According to an article on the mathematician Paul Halmos, “in the American Scientist 56(4): 375–389, Paul Halmos argued that mathematics is a creative art, and that mathematicians should be seen as artists, not number crunchers. He discussed the division of the field into mathology and mathophysics, further arguing that mathematicians and painters think and work in related ways”. By the way, Paul Halmos is a fascinating free mind. I like very much his… “Don’t just read it; fight it! Ask your own questions, look for your own examples, discover your own proofs. Is the hypothesis necessary? Is the converse true? What happens in the classical special case? What about the degenerate cases? Where does the proof use the hypothesis?”
Personally, I believe that we should not only view physics as an application of mathematics. For me, mathematics is an autonomous self-consistent aesthetic abstraction of physics, which goes beyond the physical world, and which is intimately based on the human reasoning. If you like philosophy, you may take a look at the numerous books around platonism and anti-platonism. There is a famous article by Eugene Wigner (Nobel Prize in Physics) entitled The Unreasonable Effectiveness of Mathematics in the Natural Sciences (1960) and an article by Michael Atiyah (Fields medalist) entitled On the unreasonable effectiveness of physics in mathematics (2002). You may also enjoy reading The unreasonable effectiveness of mathematics (1980) by Richard Hamming as well as The Reasonable Ineffectiveness of Mathematics (2013) by Derek Abbott. If you read French, you may enjoy the reading of Triangle de pensées.
About platonism in mathematics, you may think about this excerpt from a letter of Charles Hermite to Thomas Stieltjes : «Je vous ferais bondir, si j’osais vous avouer que je n’admets aucune solution de continuité, aucune coupure entre les mathématiques et la physique, et que les nombres entiers me semblent exister en dehors de nous et en s’imposant avec la même nécessité, la même fatalité que le sodium, le potassium, etc.»
I do not resist to quote Bertrand Russell in The Study of Mathematics (1902): “Mathematics, rightly viewed, possesses not only truth, but supreme beauty – a beauty cold and austere, like that of sculpture, without appeal to any part of our weaker nature, without the gorgeous trappings of painting or music, yet sublimely pure, and capable of a stern perfection such as only the greatest art can show”. As you can see, this smart English logician is much colder than the American Hungarian Paul Halmos. Mathematicians are as diverse than mathematics…
Let us end up with Sonya Kovalevsky, who said “Many who have never had occasion to learn what mathematics is confuse it with arithmetic, and consider it a dry and arid science. In reality, however, it is the science which demands the utmost imagination [which is more than just making things up] … It seems to me that the poet must see what others do not see, must look deeper than others look. And the mathematician must do the same thing. As for myself, all my life I have been unable to decide for which I had the greater inclination, mathematics or literature.”.