是否有一些现成的库或包用于python或R,以减少大型分类因素的级别数？

我想实现类似于R: “Binning” categorical variables的东西,但编码成最常见的top-k因子和“other”.

解决方法:

这是R中使用data.table的一个例子,但是如果没有data.table也应该很容易.

# Load data.table
require(data.table)

# Some data
set.seed(1)
dt <- data.table(type = factor(sample(c("A", "B", "C"), 10e3, replace = T)),
                 weight = rnorm(n = 10e3, mean = 70, sd = 20))

# Decide the minimum frequency a level needs...
min.freq <- 3350

# Levels that don't meet minumum frequency (using data.table)
fail.min.f <- dt[, .N, type][N < min.freq, type]

# Call all these level "Other"
levels(dt$type)[fail.min.f] <- "Other"