bugGneural Network - Bugs: bug #48923, Simulated Annealing algorithm is...

 
 

bug #48923: Simulated Annealing algorithm is just plain wrong

Submitted by:  Ray Dillinger <rayd>
Submitted on:  Sat 27 Aug 2016 04:16:43 PM UTC  
 
Category: NoneSeverity: 4 - Important
Item Group: NoneStatus: None
Privacy: PublicAssigned to: None
Open/Closed: Open

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Sat 27 Aug 2016 04:16:43 PM UTC, original submission:

While reviewing the code to see how to change the underlying representation of neural networks, I observed that the simulated annealing test case in modulus_of_two_dimensional_vectors.input does not work. On investigation I discovered that the code in simulated_annealing.c does not implement simulated annealing.

It most resembles simulated annealing as implemented in classic search. Simulated annealing in classic search accepts or rejects small random modifications of a solution, exploring gradients local to the best solution/s so far discovered. This code accepts or rejects independent random guesses at a solution using no information from the best solution/s so far discovered.

The expected running time in simulated annealing is proportional to the length of the gradient to be climbed and limited by the schedule. The expected running time between every discovered improvement with the current algorithm is exponential to the number of previously discovered improvements.

In neural networks, Simulated Annealing as implemented in classic search can be used as an unsupervised learning algorithm, and is worth implementing for that reason. But the term is also (and usually) used to refer to a schedule for reducing the learning rate in some other algorithm such as steepest gradient descent.

Learning rate schedules are also important to implement, but if we include the option to do classic-search simulated annealing, then they should be called "Learning Rate Schedules" instead.

Ray Dillinger <rayd>
Project Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by rayd (Submitted the item)
  • -unavailable- added by rayd
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follows 1 latest change.

    Date Changed By Updated Field Previous Value => Replaced By
    Sat 27 Aug 2016 04:16:43 PM UTCraydCarbon-Copy-=>Added -unavailable-

    Back to the top


    Powered by Savane 3.1-cleanup1