The South Americans are joined in the knock-out stages by Greece, who reach the last 16 of a World Cup for the first time in their history after a stoppage time goal sealed a 2-1 victory over Ivory Coast.
In Cuiaba, Japan's hopes receded as early as the 17th minute when Juan Cuadrado scored a penalty following Yasuyuki Konno's foul on Adrian Ramos.
James Rodriguez celebrates his goal against Japan
Shinji Okazaki scored an unlikely equaliser for Japan on the stroke of half time but they were blown away by Colombia after the break.
Jackson Martinez restored Colombia's lead ten minutes after the restart and he proceeded to put the game beyond Japan's reach with his second goal seven minutes from time.
And Colombia sealed their impressive victory with a goal from the outstanding James Rodriguez.
Over in Fortaleza, Ivory Coast went into their game as favourites to secure second spot in the group behind Colombia.
And the African side were poised to go through before Celtic's Georgios Samaras scored a dramatic late winner from the penalty spot.
Greece had already thrown away the lead after an Andreas Samaris goal in the first half was cancelled out by Swansea striker Wilfried Bony's equaliser, which looked like it would be enough to secure second place in the Group.
But when the referee decided Giovanni Sio had impeded Samaras one minute into stoppage time, he pointed to the spot and awarded a penalty.
Greece will now play against Costa Rica in the next round.
