Diego Simeone's side looked sharp as they drew 1-1 with Valencia in their opening fixture but were sluggish throughout the first half against their local rivals - Thomas Lemar's low effort the only chance of note before VAR dismissed Diego Costa's claim for a penalty.
And though Costa could have done better from a Lemar cross early in the second period, it was Rayo who were looking the most likely to break the deadlock.
But Griezmann, who still appears to be far from his best after winning the World Cup with France, was on the spot to tuck home from close range in the 63rd minute, though Jan Oblak had to be at his best late on for Atleti to secure the spoils.
@AntoGriezmann sure does like the Wanda @Metropolitano.— Atlético de Madrid (@atletienglish) August 25, 2018
First goal at the Wanda Metropolitano in the 2017/18 season
First goal at the Wanda Metropolitano in the 2018/19 season
#AúpaAtleti #AtletiRayo pic.twitter.com/H0rcoeFnRG
It took until the 37th minute for Atleti to muster a meaningful attack, Lemar drawing a good save from Alberto Garcia with a shot from the edge of the box.
Costa cut a frustrated figure up top, but the former Chelsea man wanted a penalty when he flicked the ball into Oscar Trejo's arm. Referee Jose Luis Gonzalez Gonzalez rightly waved away the appeal with the help of video technology.
Lemar nearly turned provider after the second-half restart with a neat cutback towards Costa, but the forward was unable to prod home from six yards.
Griezmann suffered no such profligacy, though, when he reacted sharply to get on the end of Stefan Savic's knock-down and hooked in his first goal of the season.
But it was Oblak who proved Atleti's hero late on, as the goalkeeper pulled off a string of superb saves to keep Rayo at bay and seal the points.
What does it mean: Blunt Atleti relying on instinct
Atleti looked as though they would be mounting a serious title challenge when they swiped aside Real Madrid in the UEFA Super Cup, but based on Saturday's evidence, Los Blancos and Barcelona will not have too much to worry about this campaign. Simeone's side looked bereft of ideas at times, ultimately relying on a slice of luck and the sharp reactions of Griezmann to see them clear against a Rayo side that lost 4-1 at home to Sevilla in their opening game.
Pat on the back: Oblak at his best when it mattered
It was not the best day for Atleti's front line, but with a goalkeeper like Oblak one goal is often likely to be enough. Adrian Embarba, Jose Pozo and Sergio Moreno all thought they had snatched an equaliser in the closing stages, but each time Oblak came out on top to keep Atleti's clean sheet intact.
Boot up the backside: Griezmann's World Cup hangover continues
Griezmann was one of France's stand-out performers in Russia, but the forward - who presented the World Cup trophy to the Wanda Metropolitano crowd prior to kick-off - put in another uncharacteristically poor display against Rayo. His goal proved just how vital he is to Simeone's side and he will now be hoping to kick on, but there is plenty of room for improvement from Atleti's star man.
Key Opta facts:
32 - Antoine Griezmann has been involved in 32 goals in 32 appearances for Atletico in 2018 in all competitions (23 goals and nine assists). Gold. pic.twitter.com/8h8ltlxbLv— OptaJose (@OptaJose) August 25, 2018
- Oblak has saved 11 of the last 14 shots on target he has faced in LaLiga (three from three versus Rayo).
- Griezmann has been involved in 32 goals in 32 appearances for Atletico in 2018 in all competitions (23 goals and nine assists).
- Costa has failed to score in his last 11 league games, his worst goalscoring run since March 2011 (15 in LaLiga and the Premier League).
- Rayo have failed to win in their last 18 LaLiga away games (D7 L11), their worst run since April 2002 (22).
- Thomas Partey has become the first African player to make 100 appearances for Atletico in all competitions in the 21st century.
Atleti will look to put in a more convincing display when they travel to Celta Vigo, while Rayo welcome Athletic Bilbao to the capital.