See doing a PhD in Environment Science don't affect your career ahead, if you want to seek Assistant Professor positions in Colleges for the Department of Physics. If you are aiming for the professorship in Universities, then it may create problem regarding the expertise. Although, you have good scope to apply for the Department of Chemistry or any university where there is quite a exposure to your field.

So for colleges (UG) you will not face much of a problem but for University positions it may create some issues.

For more information regarding the career in Environmental Science you can consider the link below:

https://www.careers360.com/courses/environmental-studies-course

I hope this helps. Wish you a great eve!