Correct manners are very important among the Japanese. As a foreigner in Japan you should be familiar with at least the most basic rules:
Never enter a house with your shoes on. This is one of the few rules for which Japanese will not make allowance just because you are a foreigner. This rule is also valid for some establishments like schools, some restaurants and even in changing rooms in clothes shops. Slippers are usually provided in the entrance hall. If slippers are provided for the toilet, use them instead of the ones or the rest of the house.
When you are invited into a Japanese family, bring a small present or "omiyage" (souvenir, usually food). If you are coming straight from your country, it is preferable to bring some local culinary specialties from your home town/region.
Most shops, cafes or department stores will provide plastic covers for umbrellas or somewhere to leave them outside. Make sure not to enter with a dripping wet umbrella. Refrain from blowing your nose in front of other people. Japanese only use paper tissue for this. Like in other Asian countries, it is considered rude to blow your nose in a handkerchief and stuff it in your pocket afterward.
Do not point your finger, feet or chopsticks at people. If you have to indicate an object or direction to someone, wave your fingers with the palm downwards.
Avoid expressing your opinion too directly. Japanese have what they call "honne" (real opinion) and "tatemae" (public opinion). They will express the latter in most situations so as not to disturb the group harmony. This is the reason why Japanese are so bad at debating serious issues in public. Do not interrupt people when they are speaking or while they are thinking about an answer. Japanese do not mind short periods of silence in the middle of a discussion.
Do not use your mobile phone in trains unless it is clearly stated that you are allowed to do so. Using emails or SMS is fine though.
Do not stick your chopsticks vertically into a bowl of rice, as this is used in Buddhist funerary ceremony and is considered to bring bad luck. At a "nomikai" (e.g. while going drinking with colleagues), you should (re)fill the glasses of people around you when they are empty, and they should do the same for you. If you want to refill your glass, start by serving other people. If you do not want a refill, do not empty your glass. It is polite to say "itadakimasu" once before eating or drinking, and "gochisousama deshita" to your host or to the restaurant's staff after eating or when leaving the establishment.
Contrarily to Western manners, noodles can be and should be slurped. Likewise, bowls or plates should be brought up the the mouth rather than bending one's head toward it.
"Meishi" (business cards) are exchanged when meeting someone for the first time. They should be given and accepted with both hands in formal situations. Make sure to observe it carefully and remember your opposite's name. Place the card on the table in front of you if you are sitting, or put it in your wallet.
Japanese wash themselves before entering the bath, as they have a custom of sharing the bath water. This is true as well for public baths (sento) , thermal springs (onsen) and a bath in individual homes. The reason is that other people will use the same water after you, so make sure you don't empty the bath after using it. In public baths, do not mistake men and women's changing rooms, as it is extremely impolite, even if you really mistook. The men's room are usually on the left, and normally has a blue curtain with "otoko". The women's room is usually on the right, with a red curtain reading "onna". Tattoos are banned in most public baths. If you have one, you should consult the staff at reception beforehand to avoid causing trouble.