Logistic回归的实例

Logistic回归的使用和缺失值的处理

从疝气病预测病马的死亡率

数据集:

UCI上的数据,368个样本,28个特征

测试方法:

交叉测试

实现细节:

1.数据中因为存在缺失值所以要进行预处理,这点待会再单独谈
2.数据中本来有三个标签,这里为了简单直接将未能存活和安乐死合并了
3.代码中计算10次求均值

缺失值的处理:

一般来说有这么几种方法处理缺失值:

  • 人工填写缺失值
  • 使用全局变量填充缺失值
  • 忽略有缺失值的样本
  • 使用属性的中心度量(均值或中位数等)填充缺失值
  • 使用与给定元祖同一类的所有样本的属性均值或中位数
  • 使用最可能的值(需要机器学习算法推到)
    对不同的数据我们要采用不同的方法,这里考虑到我们用Logistic回归那么我们可以采用0填充,因为用0在更新weight = weight + alpha * error * dataMatrix[randIndex]的时候不会产生更新,并且sigmoid(0)=0.5,他对结果也不会产生影响。
  • #coding=utf-8
    from numpy import *

    def loadDataSet():
        dataMat = []
        labelMat = []
        fr = open(‘testSet.txt’)
        for line in fr.readlines():
            lineArr = line.strip().split()
            dataMat.append([1.0, float(lineArr[0]), float(lineArr[1])])
            labelMat.append(int(lineArr[2]))
        return dataMat, labelMat
       
    def sigmoid(inX):
        return 1.0/(1+exp(-inX))
       
    def stocGradAscent1(dataMatrix, classLabels, numIter=150):
        m,n = shape(dataMatrix)
       
        #alpha = 0.001
        weight = ones(n)
        for j in range(numIter):
            dataIndex = range(m)
            for i in range(m):
                alpha = 4/ (1.0+j+i) +0.01
                randIndex = int(random.uniform(0,len(dataIndex)))
                h = sigmoid(sum(dataMatrix[randIndex]*weight))
                error = classLabels[randIndex] – h
                weight = weight + alpha * error * dataMatrix[randIndex]
                del(dataIndex[randIndex])
        return weight

    def classifyVector(inX, weights):
        prob = sigmoid(sum(inX*weights))
        if prob > 0.5: return 1.0
        else: return 0.0

    def colicTest():
        frTrain = open(‘horseColicTraining.txt’); frTest = open(‘horseColicTest.txt’)
        trainingSet = []; trainingLabels = []
        for line in frTrain.readlines():
            currLine = line.strip().split(‘t’)
            lineArr =[]
            for i in range(21):
                lineArr.append(float(currLine[i]))
            trainingSet.append(lineArr)
            trainingLabels.append(float(currLine[21]))
        trainWeights = stocGradAscent1(array(trainingSet), trainingLabels, 1000)
        errorCount = 0; numTestVec = 0.0
        for line in frTest.readlines():
            numTestVec += 1.0
            currLine = line.strip().split(‘t’)
            lineArr =[]
            for i in range(21):
                lineArr.append(float(currLine[i]))
            if int(classifyVector(array(lineArr), trainWeights))!= int(currLine[21]):
                errorCount += 1
        errorRate = (float(errorCount)/numTestVec)
        print “the error rate of this test is: %f” % errorRate
        return errorRate

    def multiTest():
        numTests = 10; errorSum=0.0
        for k in range(numTests):
            errorSum += colicTest()
        print “after %d iterations the average error rate is: %f” % (numTests, errorSum/float(numTests))
               
    def main():
        multiTest()
       
    if __name__ == ‘__main__’:
        main()

转载自演道,想查看更及时的互联网产品技术热点文章请点击http://go2live.cn