[Machine Learning] Exam 8 (Week 9)

해당 내용은 Andrew Ng 교수님의 Machine Learning 강의(Coursera)를 정리한 내용입니다.

 Exam 8에서 구현해야 하는 과제는 다음과 같습니다.

 

- estimateGaussian.m : 가우시안 분포를 갖는 모델의 평균과 분산을 반환하는 코드

- selectThreshold.m : Anomaly Detection에서의 최적의 Threshold를 찾는 코드

- cofiCostFunc.m : Collaborative filtering의 Cost Function 작성

 

[estimateGaussian.m]

가우시안 분포를 갖는 X 행렬의 평균과 분산을 구하면 됩니다. 평균과 분산을 구하는 공식은 아래와 같습니다.

전체 코드입니다.

function [mu sigma2] = estimateGaussian(X)
%ESTIMATEGAUSSIAN This function estimates the parameters of a 
%Gaussian distribution using the data in X
%   [mu sigma2] = estimateGaussian(X), 
%   The input X is the dataset with each n-dimensional data point in one row
%   The output is an n-dimensional vector mu, the mean of the data set
%   and the variances sigma^2, an n x 1 vector
% 

% Useful variables
[m, n] = size(X);

% You should return these values correctly
mu = zeros(n, 1);
sigma2 = zeros(n, 1);

% ====================== YOUR CODE HERE ======================
% Instructions: Compute the mean of the data and the variances
%               In particular, mu(i) should contain the mean of
%               the data for the i-th feature and sigma2(i)
%               should contain variance of the i-th feature.
%

mu = (1/m)*sum(X', 2);
sigma2 = (1/m)*sum((X' - mu).^2, 2);

% =============================================================

end

 

 

[selectThreshold.m]

Anomaly Detection에서 최적의 Threshold를 찾는데, F1-score 방법을 사용해서 찾습니다.

F1-Score에 관한 자세한 내용은 아래 게시글을 참조하시길 바랍니다.

2020/08/20 - [ML and DL/Machine Learning] - [Machine Learning] Machine Learning System Design

 

위 그림처럼 True positive, False positive, False negative를 각각 구해서 precision, recall을 구해서 F1-Score를 구하면 됩니다.

전체코드는 다음과 같습니다.

function [bestEpsilon bestF1] = selectThreshold(yval, pval)
%SELECTTHRESHOLD Find the best threshold (epsilon) to use for selecting
%outliers
%   [bestEpsilon bestF1] = SELECTTHRESHOLD(yval, pval) finds the best
%   threshold to use for selecting outliers based on the results from a
%   validation set (pval) and the ground truth (yval).
%

bestEpsilon = 0;
bestF1 = 0;
F1 = 0;

stepsize = (max(pval) - min(pval)) / 1000;
for epsilon = min(pval):stepsize:max(pval)
    
    % ====================== YOUR CODE HERE ======================
    % Instructions: Compute the F1 score of choosing epsilon as the
    %               threshold and place the value in F1. The code at the
    %               end of the loop will compare the F1 score for this
    %               choice of epsilon and set it to be the best epsilon if
    %               it is better than the current choice of epsilon.
    %               
    % Note: You can use predictions = (pval < epsilon) to get a binary vector
    %       of 0's and 1's of the outlier predictions


    % true-positive
    tp = sum(yval == 1 & pval < epsilon);

    % false-positive
    fp = sum(yval == 0 & pval < epsilon);

    % false-negative
    fn = sum(yval == 1 & pval >= epsilon);

    precison = tp / (tp + fp);
    recall = tp / (tp + fn);
    
    F1 = (2 * precison * recall) / (precison + recall);
    % =============================================================

    if F1 > bestF1
       bestF1 = F1;
       bestEpsilon = epsilon;
    end
end

end

true positive는 yval(실제 측정값)이 positive면서 예측값이 epsilon보다 작은(1으로 예측되는 것) 것의 개수이며, 나머지도 각 특성에 맞게 개수를 계산하였다.

 

[cofiCostFunc.m]

Collaborative filtering의 Cost Function을 작성하는 코드입니다. Cost Function은 다음과 같습니다.

그리고 J에 대해서 편미분한 식들은 다음과 같습니다.

전체 코드 입니다.

function [J, grad] = cofiCostFunc(params, Y, R, num_users, num_movies, ...
                                  num_features, lambda)
%COFICOSTFUNC Collaborative filtering cost function
%   [J, grad] = COFICOSTFUNC(params, Y, R, num_users, num_movies, ...
%   num_features, lambda) returns the cost and gradient for the
%   collaborative filtering problem.
%

% Unfold the U and W matrices from params
X = reshape(params(1:num_movies*num_features), num_movies, num_features);
Theta = reshape(params(num_movies*num_features+1:end), ...
                num_users, num_features);

            
% You need to return the following values correctly
J = 0;
X_grad = zeros(size(X));
Theta_grad = zeros(size(Theta));

% ====================== YOUR CODE HERE ======================
% Instructions: Compute the cost function and gradient for collaborative
%               filtering. Concretely, you should first implement the cost
%               function (without regularization) and make sure it is
%               matches our costs. After that, you should implement the 
%               gradient and use the checkCostFunction routine to check
%               that the gradient is correct. Finally, you should implement
%               regularization.
%
% Notes: X - num_movies  x num_features matrix of movie features
%        Theta - num_users  x num_features matrix of user features
%        Y - num_movies x num_users matrix of user ratings of movies
%        R - num_movies x num_users matrix, where R(i, j) = 1 if the 
%            i-th movie was rated by the j-th user
%
% You should set the following variables correctly:
%
%        X_grad - num_movies x num_features matrix, containing the 
%                 partial derivatives w.r.t. to each element of X
%        Theta_grad - num_users x num_features matrix, containing the 
%                     partial derivatives w.r.t. to each element of Theta
%
error = X*Theta' - Y;
J = (1/2)*sum(sum((error.*R).^2)) + lambda/2 * (sum(sum(Theta.^2)) + sum(sum(X.^2)));


X_grad = (error.*R) * Theta + lambda * X;
Theta_grad = (error.*R)' * X + lambda * Theta;


% =============================================================

grad = [X_grad(:); Theta_grad(:)];

end