【转】离群值的处理

2.5.3 离群值的处理
*-2.5.3.1 删除
sysuse auto, clear
adjacent price, by(foreign)
drop if (price>8814&foreign==0) |(price>9735&foreign==1)
*-or // 需要提前安装-egenmore-相关命令
sysuse auto, clear
egen outby = outside(price), by(foreign) factor(2)
drop if outby != .
*-2.5.3.2 对数转换
sysuse nlsw88, clear
gen ln_wage = ln(wage)
twoway (histogram wage,color(green)) ///
(histogram ln_wage,color(yellow))
sum wage ln_wage, d
graph box wage
graph box ln_wage
*-2.5.3.3 缩尾处理
sysuse nlsw88.dta, clear
histogram wage
*-双边缩尾
winsor wage, gen(wage_w2) p(0.025)
*-图示
twoway (histogram wage,color(green)) ///
(histogram wage_w2,color(yellow)), ///
legend(label(1 'wage') label(2 'wage_winsor2'))
*-单边缩尾

winsor wage, gen(wage_h) p(0.025) highonly
*-图示
twoway (histogram wage,color(green)) ///
(histogram wage_h,color(yellow)), ///
legend(label(1 'wage') label(2 'wage_winsorH'))
*-若无法下载 winsor 命令，可以采用如下处理方法：
_pctile wage, percentile(1 99)
replace wage = r(r1) if wage
replace wage = r(r2) if wage>r(r2)
*-亦可采用 clip() 函数 (参见第2.1.2.6小节，第385行)
gen wage_w = clip(wage, r(r1), r(r2))
sum wage wage_w, detail
*-2.5.3.4 截尾处理
sysuse nlsw88, clear
_pctile wage, percentile(1 99)
return list
drop if wage
drop if wage>r(r2) // 删除大于第99百分位的样本
*-说明：
* (1) 可以先绘制直方图，进而根据分布情况选择左截尾、
* 右截尾还是双边截尾
* (2) 相比于ln()处理和winsor处理，该处理会损失样本
* 但对于大样本而言，该方法比较“干净”