There are plenty of other states with capitols in unlikely places: Nevada, California, Washington just to name a few.
Olympia is not too oddly placed. I don't know where else you would put the capitol. I believe Sacramento is the capital city because of the gold rush, so it makes sense. Same sort of reasoning for Carson City, especially since Las Vegas only became a large city within the latter part of the century. The weirdest capital city location would have to be Florida after Alaska.
Just as Juneau was the opportune sight for the capitol thanks to the gold rush and fishing industries. Anchorage has only become a larger city in the past 30 years as well. The oldest cities in Alaska are located in the panhandle.
Lewis:
If the Northwest Angle had ferry and air services several times a day with over 30,000 inhabitants and it was chosen to be the capital city in the beginning, I wouldn't mind it much.