
Originally Posted by
MacAvoydgeen
Levenberg-Marquardt is an algorithm for optimizing the parameters of nonlinear functions. I'll give you a concrete example of how it's used. When a drug is administered to a rat, the drug gets chemically decomposed in the rat's body over a period of time. Suppose you take periodic blood samples to get a dataset of how much of the drug is left in the body at any one time. From this data, you want to produce a mathematical formula for how much of the drug is left at any one time. The formula depends on the initial dosage of the drug because, e.g., when the dosage is bigger the rat's liver might be unable to process it all or might continue to be strongly effective. So, you've collected the data at number of different dosage levels (using a number of rats) and you want a mathematical formula whereby, if I tell you I've a rat that took X micrograms, Y hours ago, you can tell me by formula how much of the drug is still left in the rat's body. The formula will be nonlinear, because drug metabolism is generally nonlinear. To come up with the formula, you GUESS that the formula is of the form: A*X*e^(B*Y), where e is 2.71818, X and Y are the dosage and time variables that you've empirically collected data for, and A and B are fixed constants whose values are unknown. The Levenberg-Marquardt algorithm will tell you the values for the unknown constants A and B in the formula that best fit the empirical data. This doesn't tell you whether the formula you GUESSED was suitable or not. So you take another guess -- maybe a formula of the form A*e^(B*X/(Y^2)) could be a closer fit. You give this new formula as the input to the Marquardt procedure, along with your empirically collected data, and the procedure returns the values for the constants A and B that optimize this new formula's fit to your data. Do that for several more formulas, using your imagination to come up with new formulas, and eventually you shall keep the formula that fits better than any of the other formulas you could think up of.
That is ALL that Marquardt-Levenberg is.
The innards of how Marquardt-Levenberg obtains its output are not easy to learn. It requires good background knowledge in numerical optimization. Most programmers have no need to know it (though I happen to know it myself). I feel very confident in guessing that the whippersnapper diddleydoo hasn't learned it at all, because if he had he wouldn't have made such a gross mistake as to associate it with genetic programming. Genetic programming is profoundly different from numerical optimization programming.