bugGneural Network - Bugs: bug #48923, Simulated Annealing algorithm is...

 
 

bug #48923: Simulated Annealing algorithm is just plain wrong

Submitter:  Ray Dillinger <rayd>
Submitted:  Sat 27 Aug 2016 04:16:43 PM UTC
   
 
Category:  None Severity:  4 - Important
Item Group:  None Status:  None
Privacy:  Public Assigned to:  None
Open/Closed:  Open
* Mandatory Fields

Add a New Comment Rich Markup
   

Sat 27 Aug 2016 04:16:43 PM UTC, original submission:  

While reviewing the code to see how to change the underlying representation of neural networks, I observed that the simulated annealing test case in modulus_of_two_dimensional_vectors.input does not work.  On investigation I discovered that the code in simulated_annealing.c does not implement simulated annealing.

It most resembles simulated annealing as implemented in classic search.  Simulated annealing in classic search accepts or rejects small random modifications of a solution, exploring gradients local to the best solution/s so far discovered.  This code accepts or rejects independent random guesses at a solution using no information from the best solution/s so far discovered.

The expected running time in simulated annealing is proportional to the length of the gradient to be climbed and limited by the schedule.  The expected running time between every discovered improvement with the current algorithm is exponential to the number of previously discovered improvements. 

In neural networks, Simulated Annealing as implemented in classic search can be used as an unsupervised learning algorithm, and is worth implementing for that reason.  But the term is also (and usually) used to refer to a schedule for reducing the learning rate in some other algorithm such as steepest gradient descent. 

Learning rate schedules are also important to implement, but if we include the option to do classic-search  simulated annealing, then they should be called "Learning Rate Schedules" instead.

Ray Dillinger <rayd>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by rayd (Submitted the item)
  • -email is unavailable- added by rayd
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follows 1 latest change.

    Date Changed by Updated Field Previous Value => Replaced by
    2016-08-27 rayd Carbon-Copy- Added -email is unavailable-

    Back to the top

    Powered by Savane 3.13-758e.
    Corresponding source code