Some interface was refined (most notably, the method for returning
Hessian was removed and the method for getting gradient was added as
optional to base Solver::Function class) and basic code for
setters/getters was added. Now is the time for the real work on an
algorithm.
This is an implementation of primal-dual algorithm, based on the C++
source code by Vadim Pisarevsky. It was extended to handle the denoising
based on multiple observations. It also contains documentation and
tests.
This is done by keeping indexToRow vector, that keeps the information,
opposite to those kept by N and B. That is, while N and B help to
determine which variable corresponds to given column in column-vector c
or row in matrix b, indexToRow helps to determine the corresponding
row/column for a given variable.
At this point, I'm waiting for comments from pull request reviewer and
not working on any upgrades. Comments are appreciated, as usual.
Use opencv's print() procedure in place of my own procedures to output
matrices and std::vectors.
Interestingly enough, operator<< does not work for matrices, when called
from my .cpp files in src/ subfolder of the optim module, although it
works when called from tests and stand-alone programs, compiled with
opencv. I think, this requires investigation and, maybe, bug report.
Attempting to fix issues pointed out by Vadim Pisarevsky during the pull
request review. In particular, the following things are done:
*) The mechanism of debug info printing is changed and made more
procedure-style than the previous macro-style
*) z in solveLP() is now returned as a column-vector
*) Func parameter of solveLP() is now allowed to be column-vector, in
which case it is understood to be the transpose of what we need
*) Func and Constr now can contain floats, not only doubles (in the
former case the conversion is done via convertTo())
*)different constructor to allocate space for z in solveLP() is used,
making the size of z more explicit (this is just a notation change, not
functional, both constructors are achieving the same goal)
*) (big) mat.hpp and iostream headers are moved to precomp-headers from
optim.hpp
In particular, the following things are done:
*) Consistent tabulation of 4 spaces is ensured
*) New function dprintf() is introduced, so now printing of the debug
information can be turned on/off via the ALEX_DEBUG macro
*) Removed solveLP_aux namespace
*) All auxiliary functions are declared as static
*) The return codes of solveLP() are encapsulated in enum.
What we have now corresponds to "formal simplex algorithm", described in
Cormen's "Intro to Algorithms". It will work *only* if the initial
problem has (0,0,0,...,0) as feasible solution (consequently, it will
work unpredictably if problem was unfeasible or did not have zero-vector as
feasible solution). Moreover, it might cycle.
TODO (first priority)
1. Implement initialize_simplex() procedure, that shall check for
feasibility and generate initial feasible solution. (in particular, code
should pass all 4 tests implemented at the moment)
2. Implement Bland's rule to avoid cycling.
3. Make the code more clear.
4. Implement several non-trivial tests (??) and check algorithm against
them. Debug if necessary.
TODO (second priority)
1. Concentrate on stability and speed (make difficult tests)
Added LPSolver class together with two nested classes: LPFunction and
LPConstraints. These represent function to be maximized and constraints
imposed respectively. They are implementations of interfaces Function
and Constraints respectively (latter ones are nested classes of Solver
interface, which is generic interface for all optimization algorithms to
be implemented within this project).
The next step is to implement the simplex algorithm! First, we shall
implement it for the case of constraints of the form Ax<=b and x>=0.
Then, we shall extend the sets of problems that can be handled by the
conversion to the one we've handled already. Finally, we shale
concentrate on numerical stability and efficiency.
At this point we have a skeleton of a new module (optim) which can
barely compile properly (unlike previous commit). Besides, there is a
first draft of solver and lpsolver (linear optimization solver) in this
commit.
