I am involved with the development of innovative software. The development is innovative since we don't know how to develop it and what algorithm should we use to implement and nobody else did it before. The process consists of several stages of studying books/papers, suggesting algorithms, writing prototypes and comparing the result with actual data. We hope that after some iteration, we converge to a valid software system.

What is the best project management approach that we can use? Is there any project management software for these types of projects?