Unless you already have connections, don't bother looking for anything under the table.
In order to work in Japan you need an appropriate work visa, which accordingly requires a sponsor (usually the company that hires you, or someone you know professionally). Without the visa, you can only enter Japan as a visitor, which prohibits work, and can only stay up to 90 days.
People have gotten away with under the table jobs on their visitor visas (or student visas), such as working at ex-pat bars or some part time jobs, but it's very risky. In addition, trying to get an apartment, cellphone, or anything that requires a contract, will require documents (first and foremost your foreigner ID card) which will state your visa status.
It is fully possible for you to enter Japan with a visitor visa, then secure a job and transfer it to a legal work visa while in Japan. But you will need to know where to look for work and how to do the interviews, etc -- which greatly depends on connections. Otherwise your best bet, without any significant professional connections, is to apply from home with an English dispatch (JET, Interac, etc) or eikaiwa company (GEOS, ECC, etc).