The Swiss failed to reach the quarter-finals of the clay-court grand slam for the first time since 2004, Gulbis emerging triumphant 6-7 (5-7) 7-6 (7-3) 6-2 4-6 6-3.
Both players won a set each on a tie-break to start the match, before 18th seed Gulbis displayed his quality by taking the third in emphatic fashion.
But 17-time grand slam champion Federer won a fourth set in which Gulbis drew the ire of the Court Philippe Chatrier crowd by leaving the court for treatment on his back with Federer 5-2 ahead.
Despite losing the fourth, Gulbis returned a revitalised player, breaking Federer's serve at the first opportunity in the decider and then confidently closing out the biggest win of his career.
But, while Federer's focus will now switch to the grass-court season and Wimbledon, world number two Novak Djokovic is still very much in contention after destroying home hope Jo-Wilfried Tsonga.
Djokovic had beaten Tsonga in an epic five-set quarter-final two years ago, but this contest proved to be nowhere near as close as Tsonga committed 38 unforced errors on his way to a 6-1 6-4 6-1 loss.
Serbian Djokovic will now face Milos Raonic in the last eight after the Canadian reached the quarter-finals of a grand slam for the first time in his career, racking up 54 winners in a 6-3 6-3 6-3 victory over Marcel Granollers of Spain.
Meanwhile, Andy Murray and Fernando Verdasco will meet in the fourth round Monday after each won contests carried over from Saturday.
Murray resumed his third-round clash with Philipp Kohlschreiber of Germany with the match tied 7-7 in the fifth, and finally brought proceedings to an end after four hours and seven minutes when he struck the crucial break to earn a 3-6 6-3 6-3 4-6 12-10 triumph.
Verdasco, by contrast, enjoyed a relatively serene route into the last 16 by completing a 6-3 6-2 6-3 success against France's Richard Gasquet.
Federer's loss opens up the draw somewhat for sixth seed Tomas Berdych, who will hope to end Gulbis' run after overcoming American John Isner 6-4 6-4 6-4
The Czech needed just three breaks in the entire match to eliminate 10th seed Isner, winning 83 percent points on his first serve to keep hopes of a second grand slam final alive.