mirror of
https://github.com/opencv/opencv.git
synced 2025-01-18 14:13:15 +08:00
Modified logistic regression module according to comments
- Reworked documentation to reflect actual code - Removed some unused variables - Removed unnecessary 'cv::' modifiers
This commit is contained in:
Maksim Shabunin
parent
4667e18831
commit
108caae216
@ -28,20 +28,22 @@ or class 0 if
|
||||
.
|
||||
|
||||
In Logistic Regression, choosing the right parameters is of utmost importance for reducing the training error and ensuring high training accuracy.
|
||||
``LogisticRegressionParams`` is the structure that defines parameters that are required to train a Logistic Regression classifier.
|
||||
The learning rate is determined by ``LogisticRegressionParams.alpha``. It determines how faster we approach the solution.
|
||||
``LogisticRegression::Params`` is the structure that defines parameters that are required to train a Logistic Regression classifier.
|
||||
The learning rate is determined by ``LogisticRegression::Params.alpha``. It determines how faster we approach the solution.
|
||||
It is a positive real number. Optimization algorithms like Batch Gradient Descent and Mini-Batch Gradient Descent are supported in ``LogisticRegression``.
|
||||
It is important that we mention the number of iterations these optimization algorithms have to run.
|
||||
The number of iterations are mentioned by ``LogisticRegressionParams.num_iters``.
|
||||
The number of iterations are mentioned by ``LogisticRegression::Params.num_iters``.
|
||||
The number of iterations can be thought as number of steps taken and learning rate specifies if it is a long step or a short step. These two parameters define how fast we arrive at a possible solution.
|
||||
In order to compensate for overfitting regularization is performed, which can be enabled by setting ``LogisticRegressionParams.regularized`` to a positive integer (greater than zero).
|
||||
One can specify what kind of regularization has to be performed by setting ``LogisticRegressionParams.norm`` to ``LogisticRegression::REG_L1`` or ``LogisticRegression::REG_L2`` values.
|
||||
``LogisticRegression`` provides a choice of 2 training methods with Batch Gradient Descent or the Mini-Batch Gradient Descent. To specify this, set ``LogisticRegressionParams.train_method`` to either ``LogisticRegression::BATCH`` or ``LogisticRegression::MINI_BATCH``.
|
||||
If ``LogisticRegressionParams`` is set to ``LogisticRegression::MINI_BATCH``, the size of the mini batch has to be to a postive integer using ``LogisticRegressionParams.mini_batch_size``.
|
||||
In order to compensate for overfitting regularization is performed, which can be enabled by setting ``LogisticRegression::Params.regularized`` to a positive integer (greater than zero).
|
||||
One can specify what kind of regularization has to be performed by setting ``LogisticRegression::Params.norm`` to ``LogisticRegression::REG_L1`` or ``LogisticRegression::REG_L2`` values.
|
||||
``LogisticRegression`` provides a choice of 2 training methods with Batch Gradient Descent or the Mini-Batch Gradient Descent. To specify this, set ``LogisticRegression::Params.train_method`` to either ``LogisticRegression::BATCH`` or ``LogisticRegression::MINI_BATCH``.
|
||||
If ``LogisticRegression::Params`` is set to ``LogisticRegression::MINI_BATCH``, the size of the mini batch has to be to a postive integer using ``LogisticRegression::Params.mini_batch_size``.
|
||||
|
||||
A sample set of training parameters for the Logistic Regression classifier can be initialized as follows:
|
||||
|
||||
::
|
||||
LogisticRegressionParams params;
|
||||
|
||||
LogisticRegression::Params params;
|
||||
params.alpha = 0.5;
|
||||
params.num_iters = 10000;
|
||||
params.norm = LogisticRegression::REG_L2;
|
||||
@ -49,16 +51,19 @@ A sample set of training parameters for the Logistic Regression classifier can b
|
||||
params.train_method = LogisticRegression::MINI_BATCH;
|
||||
params.mini_batch_size = 10;
|
||||
|
||||
**References:**
|
||||
|
||||
.. [LogRegWiki] http://en.wikipedia.org/wiki/Logistic_regression. Wikipedia article about the Logistic Regression algorithm.
|
||||
|
||||
.. [RenMalik2003] Learning a Classification Model for Segmentation. Proc. CVPR, Nice, France (2003).
|
||||
|
||||
.. [LogRegTomMitch] http://www.cs.cmu.edu/~tom/NewChapters.html. "Generative and Discriminative Classifiers: Naive Bayes and Logistic Regression" in Machine Learning, Tom Mitchell.
|
||||
|
||||
.. [BatchDesWiki] http://en.wikipedia.org/wiki/Gradient_descent_optimization. Wikipedia article about Gradient Descent based optimization.
|
||||
|
||||
LogisticRegressionParams
|
||||
------------------------
|
||||
.. ocv:struct:: LogisticRegressionParams
|
||||
LogisticRegression::Params
|
||||
--------------------------
|
||||
.. ocv:struct:: LogisticRegression::Params
|
||||
|
||||
Parameters of the Logistic Regression training algorithm. You can initialize the structure using a constructor or declaring the variable and initializing the the individual parameters.
|
||||
|
||||
@ -92,94 +97,71 @@ LogisticRegressionParams
|
||||
|
||||
Sets termination criteria for training algorithm.
|
||||
|
||||
LogisticRegressionParams::LogisticRegressionParams
|
||||
--------------------------------------------------
|
||||
The constructors.
|
||||
LogisticRegression::Params::Params
|
||||
----------------------------------
|
||||
The constructors
|
||||
|
||||
.. ocv:function:: LogisticRegressionParams::LogisticRegressionParams()
|
||||
|
||||
.. ocv:function:: LogisticRegressionParams::LogisticRegressionParams(double learning_rate, int iters, int train_method, int normlization, int reg, int mini_batch_size)
|
||||
.. ocv:function:: LogisticRegression::Params::Params(double learning_rate = 0.001, int iters = 1000, int method = LogisticRegression::BATCH, int normlization = LogisticRegression::REG_L2, int reg = 1, int batch_size = 1)
|
||||
|
||||
:param learning_rate: Specifies the learning rate.
|
||||
|
||||
:param iters: Specifies the number of iterations.
|
||||
|
||||
:param: train_method: Specifies the kind of training method used. It should be set to either ``LogisticRegression::BATCH`` or ``LogisticRegression::MINI_BATCH``. If using ``LogisticRegression::MINI_BATCH``, set ``LogisticRegressionParams.mini_batch_size`` to a positive integer.
|
||||
:param train_method: Specifies the kind of training method used. It should be set to either ``LogisticRegression::BATCH`` or ``LogisticRegression::MINI_BATCH``. If using ``LogisticRegression::MINI_BATCH``, set ``LogisticRegression::Params.mini_batch_size`` to a positive integer.
|
||||
|
||||
:param normalization: Specifies the kind of regularization to be applied. ``LogisticRegression::REG_L1`` or ``LogisticRegression::REG_L2`` (L1 norm or L2 norm). To use this, set ``LogisticRegressionParams.regularized`` to a integer greater than zero.
|
||||
:param normalization: Specifies the kind of regularization to be applied. ``LogisticRegression::REG_L1`` or ``LogisticRegression::REG_L2`` (L1 norm or L2 norm). To use this, set ``LogisticRegression::Params.regularized`` to a integer greater than zero.
|
||||
|
||||
:param: reg: To enable or disable regularization. Set to positive integer (greater than zero) to enable and to 0 to disable.
|
||||
:param reg: To enable or disable regularization. Set to positive integer (greater than zero) to enable and to 0 to disable.
|
||||
|
||||
:param: mini_batch_size: Specifies the number of training samples taken in each step of Mini-Batch Gradient Descent. Will only be used if using ``LogisticRegression::MINI_BATCH`` training algorithm. It has to take values less than the total number of training samples.
|
||||
|
||||
The full constructor initializes corresponding members. The default constructor creates an object with dummy parameters.
|
||||
|
||||
::
|
||||
|
||||
LogisticRegressionParams::LogisticRegressionParams()
|
||||
{
|
||||
term_crit = cv::TermCriteria(TermCriteria::COUNT + TermCriteria::EPS, 1000, 0.001);
|
||||
alpha = 0.001;
|
||||
num_iters = 1000;
|
||||
norm = LogisticRegression::REG_L2;
|
||||
regularized = 1;
|
||||
train_method = LogisticRegression::BATCH;
|
||||
mini_batch_size = 1;
|
||||
}
|
||||
:param mini_batch_size: Specifies the number of training samples taken in each step of Mini-Batch Gradient Descent. Will only be used if using ``LogisticRegression::MINI_BATCH`` training algorithm. It has to take values less than the total number of training samples.
|
||||
|
||||
By initializing this structure, one can set all the parameters required for Logistic Regression classifier.
|
||||
|
||||
LogisticRegression
|
||||
------------------
|
||||
.. ocv:class:: LogisticRegression
|
||||
|
||||
.. ocv:class:: LogisticRegression : public StatModel
|
||||
|
||||
Implements Logistic Regression classifier.
|
||||
|
||||
LogisticRegression::LogisticRegression
|
||||
--------------------------------------
|
||||
The constructors.
|
||||
LogisticRegression::create
|
||||
--------------------------
|
||||
Creates empty model.
|
||||
|
||||
.. ocv:function:: LogisticRegression::LogisticRegression( const LogisticRegressionParams& params = LogisticRegressionParams())
|
||||
.. ocv:function:: Ptr<LogisticRegression> LogisticRegression::create( const Params& params = Params() )
|
||||
|
||||
:param params: The training parameters for the classifier of type ``LogisticRegressionParams``.
|
||||
:param params: The training parameters for the classifier of type ``LogisticRegression::Params``.
|
||||
|
||||
.. ocv:function:: LogisticRegression::LogisticRegression(cv::InputArray data_ip, cv::InputArray labels_ip, const LogisticRegressionParams& params)
|
||||
|
||||
:param data: The data variable of type ``CV_32F``. Each data instance has to be arranged per across different rows.
|
||||
|
||||
:param labels_ip: The data variable of type ``CV_32F``. Each label instance has to be arranged across different rows.
|
||||
|
||||
:param params: The training parameters for the classifier of type ``LogisticRegressionParams``.
|
||||
|
||||
The constructor with parameters allows to create a Logistic Regression object intialized with given data and trains it.
|
||||
Creates Logistic Regression model with parameters given.
|
||||
|
||||
LogisticRegression::train
|
||||
-------------------------
|
||||
Trains the Logistic Regression classifier and returns true if successful.
|
||||
|
||||
.. ocv:function:: bool LogisticRegression::train(cv::InputArray data_ip, cv::InputArray label_ip)
|
||||
.. ocv:function:: bool LogisticRegression::train( const Ptr<TrainData>& trainData, int flags=0 )
|
||||
|
||||
:param data_ip: An InputArray variable of type ``CV_32F``. Each data instance has to be arranged per across different rows.
|
||||
|
||||
:param labels_ip: An InputArray variable of type ``CV_32F``. Each label instance has to be arranged across differnet rows.
|
||||
:param trainData: Instance of ml::TrainData class holding learning data.
|
||||
|
||||
:param flags: Not used.
|
||||
|
||||
LogisticRegression::predict
|
||||
---------------------------
|
||||
Predicts responses for input samples and returns a float type.
|
||||
|
||||
.. ocv:function:: void LogisticRegression::predict( cv::InputArray data, cv::OutputArray predicted_labels ) const
|
||||
.. ocv:function:: void LogisticRegression::predict( InputArray samples, OutputArray results=noArray(), int flags=0 ) const
|
||||
|
||||
:param data: The input data for the prediction algorithm. The ``data`` variable should be of type ``CV_32F``.
|
||||
:param samples: The input data for the prediction algorithm. Matrix [m x n], where each row contains variables (features) of one object being classified. Should have data type ``CV_32F``.
|
||||
|
||||
:param predicted_labels: Predicted labels as a column matrix and of type ``CV_32S``.
|
||||
:param results: Predicted labels as a column matrix of type ``CV_32S``.
|
||||
|
||||
:param flags: Not used.
|
||||
|
||||
|
||||
LogisticRegression::get_learnt_thetas
|
||||
-------------------------------------
|
||||
This function returns the trained paramters arranged across rows. For a two class classifcation problem, it returns a row matrix.
|
||||
|
||||
.. ocv:function:: const cv::Mat LogisticRegression::get_learnt_thetas() const
|
||||
.. ocv:function:: Mat LogisticRegression::get_learnt_thetas() const
|
||||
|
||||
It returns learnt paramters of the Logistic Regression as a matrix of type ``CV_32F``.
|
||||
|
||||
|
||||
@ -18,5 +18,5 @@ Most of the classification and regression algorithms are implemented as C++ clas
|
||||
random_trees
|
||||
expectation_maximization
|
||||
neural_networks
|
||||
mldata
|
||||
logistic_regression
|
||||
mldata
|
||||
|
||||
@ -89,9 +89,6 @@ public:
|
||||
CV_PROP_RW double maxVal;
|
||||
CV_PROP_RW double logStep;
|
||||
};
|
||||
#define CV_TYPE_NAME_ML_LR "opencv-ml-lr"
|
||||
|
||||
|
||||
|
||||
class CV_EXPORTS TrainData
|
||||
{
|
||||
@ -590,7 +587,7 @@ public:
|
||||
int regularized;
|
||||
int train_method;
|
||||
int mini_batch_size;
|
||||
cv::TermCriteria term_crit;
|
||||
TermCriteria term_crit;
|
||||
};
|
||||
|
||||
enum { REG_L1 = 0, REG_L2 = 1};
|
||||
|
||||
@ -73,7 +73,7 @@ LogisticRegression::Params::Params(double learning_rate,
|
||||
regularized = reg;
|
||||
train_method = method;
|
||||
mini_batch_size = batch_size;
|
||||
term_crit = cv::TermCriteria(TermCriteria::COUNT + TermCriteria::EPS, num_iters, alpha);
|
||||
term_crit = TermCriteria(TermCriteria::COUNT + TermCriteria::EPS, num_iters, alpha);
|
||||
}
|
||||
|
||||
class LogisticRegressionImpl : public LogisticRegression
|
||||
@ -90,25 +90,25 @@ public:
|
||||
virtual void clear();
|
||||
virtual void write(FileStorage& fs) const;
|
||||
virtual void read(const FileNode& fn);
|
||||
virtual cv::Mat get_learnt_thetas() const;
|
||||
virtual Mat get_learnt_thetas() const;
|
||||
virtual int getVarCount() const { return learnt_thetas.cols; }
|
||||
virtual bool isTrained() const { return !learnt_thetas.empty(); }
|
||||
virtual bool isClassifier() const { return true; }
|
||||
virtual String getDefaultModelName() const { return "opencv_ml_lr"; }
|
||||
protected:
|
||||
cv::Mat calc_sigmoid(const cv::Mat& data) const;
|
||||
double compute_cost(const cv::Mat& _data, const cv::Mat& _labels, const cv::Mat& _init_theta);
|
||||
cv::Mat compute_batch_gradient(const cv::Mat& _data, const cv::Mat& _labels, const cv::Mat& _init_theta);
|
||||
cv::Mat compute_mini_batch_gradient(const cv::Mat& _data, const cv::Mat& _labels, const cv::Mat& _init_theta);
|
||||
bool set_label_map(const cv::Mat& _labels_i);
|
||||
cv::Mat remap_labels(const cv::Mat& _labels_i, const map<int, int>& lmap) const;
|
||||
Mat calc_sigmoid(const Mat& data) const;
|
||||
double compute_cost(const Mat& _data, const Mat& _labels, const Mat& _init_theta);
|
||||
Mat compute_batch_gradient(const Mat& _data, const Mat& _labels, const Mat& _init_theta);
|
||||
Mat compute_mini_batch_gradient(const Mat& _data, const Mat& _labels, const Mat& _init_theta);
|
||||
bool set_label_map(const Mat& _labels_i);
|
||||
Mat remap_labels(const Mat& _labels_i, const map<int, int>& lmap) const;
|
||||
protected:
|
||||
Params params;
|
||||
cv::Mat learnt_thetas;
|
||||
Mat learnt_thetas;
|
||||
map<int, int> forward_mapper;
|
||||
map<int, int> reverse_mapper;
|
||||
cv::Mat labels_o;
|
||||
cv::Mat labels_n;
|
||||
Mat labels_o;
|
||||
Mat labels_n;
|
||||
};
|
||||
|
||||
Ptr<LogisticRegression> LogisticRegression::create(const Params& params)
|
||||
@ -119,8 +119,8 @@ Ptr<LogisticRegression> LogisticRegression::create(const Params& params)
|
||||
bool LogisticRegressionImpl::train(const Ptr<TrainData>& trainData, int)
|
||||
{
|
||||
clear();
|
||||
cv::Mat _data_i = trainData->getSamples();
|
||||
cv::Mat _labels_i = trainData->getResponses();
|
||||
Mat _data_i = trainData->getSamples();
|
||||
Mat _labels_i = trainData->getResponses();
|
||||
|
||||
CV_Assert( !_labels_i.empty() && !_data_i.empty());
|
||||
|
||||
@ -140,14 +140,14 @@ bool LogisticRegressionImpl::train(const Ptr<TrainData>& trainData, int)
|
||||
|
||||
bool ok = false;
|
||||
|
||||
cv::Mat labels;
|
||||
Mat labels;
|
||||
|
||||
set_label_map(_labels_i);
|
||||
int num_classes = (int) this->forward_mapper.size();
|
||||
|
||||
// add a column of ones
|
||||
cv::Mat data_t = cv::Mat::zeros(_data_i.rows, _data_i.cols+1, CV_32F);
|
||||
vconcat(cv::Mat(_data_i.rows, 1, _data_i.type(), Scalar::all(1.0)), data_t.col(0));
|
||||
Mat data_t = Mat::zeros(_data_i.rows, _data_i.cols+1, CV_32F);
|
||||
vconcat(Mat(_data_i.rows, 1, _data_i.type(), Scalar::all(1.0)), data_t.col(0));
|
||||
|
||||
for (int i=1;i<data_t.cols;i++)
|
||||
{
|
||||
@ -165,14 +165,14 @@ bool LogisticRegressionImpl::train(const Ptr<TrainData>& trainData, int)
|
||||
}
|
||||
|
||||
|
||||
cv::Mat thetas = cv::Mat::zeros(num_classes, data_t.cols, CV_32F);
|
||||
cv::Mat init_theta = cv::Mat::zeros(data_t.cols, 1, CV_32F);
|
||||
Mat thetas = Mat::zeros(num_classes, data_t.cols, CV_32F);
|
||||
Mat init_theta = Mat::zeros(data_t.cols, 1, CV_32F);
|
||||
|
||||
cv::Mat labels_l = remap_labels(_labels_i, this->forward_mapper);
|
||||
cv::Mat new_local_labels;
|
||||
Mat labels_l = remap_labels(_labels_i, this->forward_mapper);
|
||||
Mat new_local_labels;
|
||||
|
||||
int ii=0;
|
||||
cv::Mat new_theta;
|
||||
Mat new_theta;
|
||||
|
||||
if(num_classes == 2)
|
||||
{
|
||||
@ -203,7 +203,7 @@ bool LogisticRegressionImpl::train(const Ptr<TrainData>& trainData, int)
|
||||
}
|
||||
|
||||
this->learnt_thetas = thetas.clone();
|
||||
if( cvIsNaN( (double)cv::sum(this->learnt_thetas)[0] ) )
|
||||
if( cvIsNaN( (double)sum(this->learnt_thetas)[0] ) )
|
||||
{
|
||||
CV_Error( CV_StsBadArg, "check training parameters. Invalid training classifier" );
|
||||
}
|
||||
@ -215,7 +215,7 @@ float LogisticRegressionImpl::predict(InputArray samples, OutputArray results, i
|
||||
{
|
||||
/* returns a class of the predicted class
|
||||
class names can be 1,2,3,4, .... etc */
|
||||
cv::Mat thetas, data, pred_labs;
|
||||
Mat thetas, data, pred_labs;
|
||||
data = samples.getMat();
|
||||
|
||||
// check if learnt_mats array is populated
|
||||
@ -229,12 +229,12 @@ float LogisticRegressionImpl::predict(InputArray samples, OutputArray results, i
|
||||
}
|
||||
|
||||
// add a column of ones
|
||||
cv::Mat data_t = cv::Mat::zeros(data.rows, data.cols+1, CV_32F);
|
||||
Mat data_t = Mat::zeros(data.rows, data.cols+1, CV_32F);
|
||||
for (int i=0;i<data_t.cols;i++)
|
||||
{
|
||||
if(i==0)
|
||||
{
|
||||
vconcat(cv::Mat(data.rows, 1, data.type(), Scalar::all(1.0)), data_t.col(i));
|
||||
vconcat(Mat(data.rows, 1, data.type(), Scalar::all(1.0)), data_t.col(i));
|
||||
continue;
|
||||
}
|
||||
vconcat(data.col(i-1), data_t.col(i));
|
||||
@ -250,10 +250,10 @@ float LogisticRegressionImpl::predict(InputArray samples, OutputArray results, i
|
||||
Point min_loc;
|
||||
Point max_loc;
|
||||
|
||||
cv::Mat labels;
|
||||
cv::Mat labels_c;
|
||||
cv::Mat temp_pred;
|
||||
cv::Mat pred_m = cv::Mat::zeros(data_t.rows, thetas.rows, data.type());
|
||||
Mat labels;
|
||||
Mat labels_c;
|
||||
Mat temp_pred;
|
||||
Mat pred_m = Mat::zeros(data_t.rows, thetas.rows, data.type());
|
||||
|
||||
if(thetas.rows == 1)
|
||||
{
|
||||
@ -269,7 +269,7 @@ float LogisticRegressionImpl::predict(InputArray samples, OutputArray results, i
|
||||
for(int i = 0;i<thetas.rows;i++)
|
||||
{
|
||||
temp_pred = calc_sigmoid(data_t * thetas.row(i).t());
|
||||
cv::vconcat(temp_pred, pred_m.col(i));
|
||||
vconcat(temp_pred, pred_m.col(i));
|
||||
}
|
||||
for(int i = 0;i<pred_m.rows;i++)
|
||||
{
|
||||
@ -287,32 +287,30 @@ float LogisticRegressionImpl::predict(InputArray samples, OutputArray results, i
|
||||
return 0;
|
||||
}
|
||||
|
||||
cv::Mat LogisticRegressionImpl::calc_sigmoid(const cv::Mat& data) const
|
||||
Mat LogisticRegressionImpl::calc_sigmoid(const Mat& data) const
|
||||
{
|
||||
cv::Mat dest;
|
||||
cv::exp(-data, dest);
|
||||
Mat dest;
|
||||
exp(-data, dest);
|
||||
return 1.0/(1.0+dest);
|
||||
}
|
||||
|
||||
double LogisticRegressionImpl::compute_cost(const cv::Mat& _data, const cv::Mat& _labels, const cv::Mat& _init_theta)
|
||||
double LogisticRegressionImpl::compute_cost(const Mat& _data, const Mat& _labels, const Mat& _init_theta)
|
||||
{
|
||||
int llambda = 0;
|
||||
int m;
|
||||
int n;
|
||||
double cost = 0;
|
||||
double rparameter = 0;
|
||||
cv::Mat gradient;
|
||||
cv::Mat theta_b;
|
||||
cv::Mat theta_c;
|
||||
cv::Mat d_a;
|
||||
cv::Mat d_b;
|
||||
Mat theta_b;
|
||||
Mat theta_c;
|
||||
Mat d_a;
|
||||
Mat d_b;
|
||||
|
||||
m = _data.rows;
|
||||
n = _data.cols;
|
||||
|
||||
gradient = cv::Mat::zeros( _init_theta.rows, _init_theta.cols, _init_theta.type());
|
||||
theta_b = _init_theta(Range(1, n), Range::all());
|
||||
cv::multiply(theta_b, theta_b, theta_c, 1);
|
||||
multiply(theta_b, theta_b, theta_c, 1);
|
||||
|
||||
if(this->params.regularized > 0)
|
||||
{
|
||||
@ -321,31 +319,31 @@ double LogisticRegressionImpl::compute_cost(const cv::Mat& _data, const cv::Mat&
|
||||
|
||||
if(this->params.norm == LogisticRegression::REG_L1)
|
||||
{
|
||||
rparameter = (llambda/(2*m)) * cv::sum(theta_b)[0];
|
||||
rparameter = (llambda/(2*m)) * sum(theta_b)[0];
|
||||
}
|
||||
else
|
||||
{
|
||||
// assuming it to be L2 by default
|
||||
rparameter = (llambda/(2*m)) * cv::sum(theta_c)[0];
|
||||
rparameter = (llambda/(2*m)) * sum(theta_c)[0];
|
||||
}
|
||||
|
||||
d_a = calc_sigmoid(_data* _init_theta);
|
||||
|
||||
|
||||
cv::log(d_a, d_a);
|
||||
cv::multiply(d_a, _labels, d_a);
|
||||
log(d_a, d_a);
|
||||
multiply(d_a, _labels, d_a);
|
||||
|
||||
d_b = 1 - calc_sigmoid(_data * _init_theta);
|
||||
cv::log(d_b, d_b);
|
||||
cv::multiply(d_b, 1-_labels, d_b);
|
||||
log(d_b, d_b);
|
||||
multiply(d_b, 1-_labels, d_b);
|
||||
|
||||
cost = (-1.0/m) * (cv::sum(d_a)[0] + cv::sum(d_b)[0]);
|
||||
cost = (-1.0/m) * (sum(d_a)[0] + sum(d_b)[0]);
|
||||
cost = cost + rparameter;
|
||||
|
||||
return cost;
|
||||
}
|
||||
|
||||
cv::Mat LogisticRegressionImpl::compute_batch_gradient(const cv::Mat& _data, const cv::Mat& _labels, const cv::Mat& _init_theta)
|
||||
Mat LogisticRegressionImpl::compute_batch_gradient(const Mat& _data, const Mat& _labels, const Mat& _init_theta)
|
||||
{
|
||||
// implements batch gradient descent
|
||||
if(this->params.alpha<=0)
|
||||
@ -361,11 +359,11 @@ cv::Mat LogisticRegressionImpl::compute_batch_gradient(const cv::Mat& _data, con
|
||||
int llambda = 0;
|
||||
double ccost;
|
||||
int m, n;
|
||||
cv::Mat pcal_a;
|
||||
cv::Mat pcal_b;
|
||||
cv::Mat pcal_ab;
|
||||
cv::Mat gradient;
|
||||
cv::Mat theta_p = _init_theta.clone();
|
||||
Mat pcal_a;
|
||||
Mat pcal_b;
|
||||
Mat pcal_ab;
|
||||
Mat gradient;
|
||||
Mat theta_p = _init_theta.clone();
|
||||
m = _data.rows;
|
||||
n = _data.cols;
|
||||
|
||||
@ -393,7 +391,7 @@ cv::Mat LogisticRegressionImpl::compute_batch_gradient(const cv::Mat& _data, con
|
||||
|
||||
pcal_b = _data(Range::all(), Range(0,1));
|
||||
|
||||
cv::multiply(pcal_a, pcal_b, pcal_ab, 1);
|
||||
multiply(pcal_a, pcal_b, pcal_ab, 1);
|
||||
|
||||
gradient.row(0) = ((float)1/m) * sum(pcal_ab)[0];
|
||||
|
||||
@ -404,9 +402,9 @@ cv::Mat LogisticRegressionImpl::compute_batch_gradient(const cv::Mat& _data, con
|
||||
{
|
||||
pcal_b = _data(Range::all(), Range(ii,ii+1));
|
||||
|
||||
cv::multiply(pcal_a, pcal_b, pcal_ab, 1);
|
||||
multiply(pcal_a, pcal_b, pcal_ab, 1);
|
||||
|
||||
gradient.row(ii) = (1.0/m)*cv::sum(pcal_ab)[0] + (llambda/m) * theta_p.row(ii);
|
||||
gradient.row(ii) = (1.0/m)*sum(pcal_ab)[0] + (llambda/m) * theta_p.row(ii);
|
||||
}
|
||||
|
||||
theta_p = theta_p - ( static_cast<double>(this->params.alpha)/m)*gradient;
|
||||
@ -414,7 +412,7 @@ cv::Mat LogisticRegressionImpl::compute_batch_gradient(const cv::Mat& _data, con
|
||||
return theta_p;
|
||||
}
|
||||
|
||||
cv::Mat LogisticRegressionImpl::compute_mini_batch_gradient(const cv::Mat& _data, const cv::Mat& _labels, const cv::Mat& _init_theta)
|
||||
Mat LogisticRegressionImpl::compute_mini_batch_gradient(const Mat& _data, const Mat& _labels, const Mat& _init_theta)
|
||||
{
|
||||
// implements batch gradient descent
|
||||
int lambda_l = 0;
|
||||
@ -433,13 +431,13 @@ cv::Mat LogisticRegressionImpl::compute_mini_batch_gradient(const cv::Mat& _data
|
||||
CV_Error( CV_StsBadArg, "number of iterations cannot be zero or a negative number" );
|
||||
}
|
||||
|
||||
cv::Mat pcal_a;
|
||||
cv::Mat pcal_b;
|
||||
cv::Mat pcal_ab;
|
||||
cv::Mat gradient;
|
||||
cv::Mat theta_p = _init_theta.clone();
|
||||
cv::Mat data_d;
|
||||
cv::Mat labels_l;
|
||||
Mat pcal_a;
|
||||
Mat pcal_b;
|
||||
Mat pcal_ab;
|
||||
Mat gradient;
|
||||
Mat theta_p = _init_theta.clone();
|
||||
Mat data_d;
|
||||
Mat labels_l;
|
||||
|
||||
if(this->params.regularized > 0)
|
||||
{
|
||||
@ -479,7 +477,7 @@ cv::Mat LogisticRegressionImpl::compute_mini_batch_gradient(const cv::Mat& _data
|
||||
|
||||
pcal_b = data_d(Range::all(), Range(0,1));
|
||||
|
||||
cv::multiply(pcal_a, pcal_b, pcal_ab, 1);
|
||||
multiply(pcal_a, pcal_b, pcal_ab, 1);
|
||||
|
||||
gradient.row(0) = ((float)1/m) * sum(pcal_ab)[0];
|
||||
|
||||
@ -488,8 +486,8 @@ cv::Mat LogisticRegressionImpl::compute_mini_batch_gradient(const cv::Mat& _data
|
||||
for(int k = 1;k<gradient.rows;k++)
|
||||
{
|
||||
pcal_b = data_d(Range::all(), Range(k,k+1));
|
||||
cv::multiply(pcal_a, pcal_b, pcal_ab, 1);
|
||||
gradient.row(k) = (1.0/m)*cv::sum(pcal_ab)[0] + (lambda_l/m) * theta_p.row(k);
|
||||
multiply(pcal_a, pcal_b, pcal_ab, 1);
|
||||
gradient.row(k) = (1.0/m)*sum(pcal_ab)[0] + (lambda_l/m) * theta_p.row(k);
|
||||
}
|
||||
|
||||
theta_p = theta_p - ( static_cast<double>(this->params.alpha)/m)*gradient;
|
||||
@ -505,15 +503,14 @@ cv::Mat LogisticRegressionImpl::compute_mini_batch_gradient(const cv::Mat& _data
|
||||
return theta_p;
|
||||
}
|
||||
|
||||
bool LogisticRegressionImpl::set_label_map(const cv::Mat &_labels_i)
|
||||
bool LogisticRegressionImpl::set_label_map(const Mat &_labels_i)
|
||||
{
|
||||
// this function creates two maps to map user defined labels to program friendly labels two ways.
|
||||
int ii = 0;
|
||||
cv::Mat labels;
|
||||
bool ok = false;
|
||||
Mat labels;
|
||||
|
||||
this->labels_o = cv::Mat(0,1, CV_8U);
|
||||
this->labels_n = cv::Mat(0,1, CV_8U);
|
||||
this->labels_o = Mat(0,1, CV_8U);
|
||||
this->labels_n = Mat(0,1, CV_8U);
|
||||
|
||||
_labels_i.convertTo(labels, CV_32S);
|
||||
|
||||
@ -534,17 +531,16 @@ bool LogisticRegressionImpl::set_label_map(const cv::Mat &_labels_i)
|
||||
{
|
||||
this->reverse_mapper[it->second] = it->first;
|
||||
}
|
||||
ok = true;
|
||||
|
||||
return ok;
|
||||
return true;
|
||||
}
|
||||
|
||||
cv::Mat LogisticRegressionImpl::remap_labels(const cv::Mat& _labels_i, const map<int, int>& lmap) const
|
||||
Mat LogisticRegressionImpl::remap_labels(const Mat& _labels_i, const map<int, int>& lmap) const
|
||||
{
|
||||
cv::Mat labels;
|
||||
Mat labels;
|
||||
_labels_i.convertTo(labels, CV_32S);
|
||||
|
||||
cv::Mat new_labels = cv::Mat::zeros(labels.rows, labels.cols, labels.type());
|
||||
Mat new_labels = Mat::zeros(labels.rows, labels.cols, labels.type());
|
||||
|
||||
CV_Assert( lmap.size() > 0 );
|
||||
|
||||
@ -615,7 +611,7 @@ void LogisticRegressionImpl::read(const FileNode& fn)
|
||||
}
|
||||
}
|
||||
|
||||
cv::Mat LogisticRegressionImpl::get_learnt_thetas() const
|
||||
Mat LogisticRegressionImpl::get_learnt_thetas() const
|
||||
{
|
||||
return this->learnt_thetas;
|
||||
}
|
||||
|
||||
@ -74,7 +74,7 @@ static bool calculateError( const Mat& _p_labels, const Mat& _o_labels, float& e
|
||||
CV_Assert(_p_labels_temp.total() == _o_labels_temp.total());
|
||||
CV_Assert(_p_labels_temp.rows == _o_labels_temp.rows);
|
||||
|
||||
accuracy = (float)cv::countNonZero(_p_labels_temp == _o_labels_temp)/_p_labels_temp.rows;
|
||||
accuracy = (float)countNonZero(_p_labels_temp == _o_labels_temp)/_p_labels_temp.rows;
|
||||
error = 1 - accuracy;
|
||||
return true;
|
||||
}
|
||||
@ -166,7 +166,7 @@ void CV_LRTest_SaveLoad::run( int /*start_from*/ )
|
||||
params1.mini_batch_size = 10;
|
||||
|
||||
// train and save the classifier
|
||||
String filename = cv::tempfile(".xml");
|
||||
String filename = tempfile(".xml");
|
||||
try
|
||||
{
|
||||
// run LR classifier train classifier
|
||||
@ -208,8 +208,8 @@ void CV_LRTest_SaveLoad::run( int /*start_from*/ )
|
||||
// check if there is any difference between computed learnt mat and retreived mat
|
||||
|
||||
float errorCount = 0.0;
|
||||
errorCount += 1 - (float)cv::countNonZero(responses1 == responses2)/responses1.rows;
|
||||
errorCount += 1 - (float)cv::sum(comp_learnt_mats)[0]/comp_learnt_mats.rows;
|
||||
errorCount += 1 - (float)countNonZero(responses1 == responses2)/responses1.rows;
|
||||
errorCount += 1 - (float)sum(comp_learnt_mats)[0]/comp_learnt_mats.rows;
|
||||
|
||||
if(errorCount>0)
|
||||
{
|
||||
|
||||
@ -58,9 +58,9 @@
|
||||
|
||||
#include <iostream>
|
||||
|
||||
#include <opencv2/core/core.hpp>
|
||||
#include <opencv2/ml/ml.hpp>
|
||||
#include <opencv2/highgui/highgui.hpp>
|
||||
#include <opencv2/core.hpp>
|
||||
#include <opencv2/ml.hpp>
|
||||
#include <opencv2/highgui.hpp>
|
||||
|
||||
using namespace std;
|
||||
using namespace cv;
|
||||
@ -78,7 +78,7 @@ static void showImage(const Mat &data, int columns, const String &name)
|
||||
|
||||
static float calculateAccuracyPercent(const Mat &original, const Mat &predicted)
|
||||
{
|
||||
return 100 * (float)cv::countNonZero(original == predicted) / predicted.rows;
|
||||
return 100 * (float)countNonZero(original == predicted) / predicted.rows;
|
||||
}
|
||||
|
||||
int main()
|
||||
|
||||
Loading…
Reference in New Issue
Block a user