Thread is a bit old now but in case you are still interested or if someone else comes along I will answer.
If you are only staying temporarily you could probably get by without internet but that being said if you did want it then you have a few options:
1. All Starbucks have free WiFI but in order to use it you have to register on their website before you connect to their WiFi. So you need access to an internet connection before you can use theirs.
2. Some AirBnB will supply a portable WiFi device. I remember an apartment I got in Shinjuku came with a free portable WiFi modem that I could take with me.
3. Just hire a portable WiFi device from one of the many suppliers. They can post it to the airport so you can pick it up from there and it comes with a return envelope to post it. Far better method than getting a data SIM for your phone IMO.
I was in Japan last October.
(Only 2 weeks)
In Tokyo, "Free" WiFi signs were everywhere. It seems the telecom people supply it.
I stayed at the Keo Plaza, and they supplied free WiFi, AND a free cell phone.
Your mileage will vary..