Baker quickly made the part his own, viewing figures for his first few years returning to a level not seen since the height of 'Dalekmania' a decade earlier. His eccentric style of dress and quirky personality (particularly his trademark long scarf and fondness for jelly babies), as well as his distinctive voice, made him an immediately recognizable figure and he quickly caught the viewing public's imagination. Baker played the Doctor for seven consecutive seasons, making him the longest-serving actor in the part. Baker himself suggested many aspects of his Doctor's personality, but the distinctive scarf was created by accident. James Acheson, the costume designer assigned to his first story, had provided far more wool than was necessary to the knitter, Begonia Pope; Pope knitted all the wool she was given. It was Baker who suggested that he wear the ridiculously long scarf, which he did once it had been shortened a bit to make it more manageable.
The Doctor played by Tom Baker (1974–81) is often regarded as the most popular of the Doctors. In polls conducted by Doctor Who Magazine, Baker has lost the "Best Doctor" category only three times: once to Sylvester McCoy in 1990 (then the incumbent Doctor), and twice to incumbent David Tennant in 2006 and 2009. Many of the stories of his era are considered to be classics of the series, including The Ark in Space, Genesis of the Daleks, The Brain of Morbius, The Deadly Assassin and The Robots of Death. Baker was also the first actor to play the Doctor to be seen in the United States. However, the violent tone of the stories produced by Letts' successor, Philip Hinchcliffe, saw the series come under heavy criticism at home from morality campaigner Mary Whitehouse. Concerns over violence during this early period led to a lightening of the tone and an erratic decline in both the popularity and quality of the series. Baker has described Hinchcliffe as "amazing" and identified that as his favorite period of his time on the series. He described Hinchcliffe's successor, Graham Williams, as "absolutely devoted" but lacking Hinchcliffe's flair. He has acknowledged that his final producer on the series, John Nathan-Turner, made changes he didn't agree with and they "did not see eye-to-eye really about very much". He said they became good friends afterwards and forgot their disagreements. Baker did suggest he may have stayed in the role for one series too many.
Baker continues to be associated with the Doctor, appearing on documentaries such as The Story of Doctor Who and Doctor Who Confidential and giving interviews about his time on the program. He reappeared as the Doctor for the 1993 charity special Dimensions in Time and audio for the PC game Doctor Who: Destiny of the Doctors. In 1996 he appraised his time on the show as the highlight of his life. He is often interviewed as part of documentaries on the extras of Doctor Who DVD releases from his era as the Doctor and has recorded DVD commentaries for many of the stories. In a 2004 interview regarding the series' revival, Baker suggested that he be cast as the Master. In a 2006 interview with The Sun newspaper, he claims that he has not watched any episodes of the new series because he "just can't be bothered". In a 2010 interview Baker said that he had not watched Tennant's performance as the Doctor but thought his Hamlet was excellent.
While Peter Davison, Colin Baker, Sylvester McCoy, and Paul McGann have all reprised their roles for audio adventures produced since the 1990s by Big Finish (and sometimes the BBC), Baker had declined to voice the Doctor until 2009, claiming that he hadn't seen a script he liked. In July 2009, the BBC announced that Baker would return to the role for a series of five audio dramas, co-starring Richard Franklin as Captain Mike Yates, which would begin release in September. The five audios comprise a single linked story under the banner title Hornets' Nest, written by well-known author Paul Magrs. He returns with a sequel to Hornets' Nest called Demon Quest. Baker has also filmed inserts for a video release of the unfinished Shada in 1992, presented the video release The Tom Baker Years (a look back at his time on the series watching short clips from his episodes) and also provided narration for several BBC audio releases of old Doctor Who stories.
In March 2011, it was announced that Baker would be returning as the Fourth Doctor initially for two series of plays for Big Finish Productions, starring alongside former companions Leela (Louise Jameson) and Romana I (Mary Tamm). The first series of six audios were released starting from January 2012. Big Finish had also arranged for Baker to record a series of stories reuniting him with Elisabeth Sladen's character Sarah Jane Smith (for which special permission was obtained from the producers of The Sarah Jane Adventures TV series), but Sladen died in April 2011 before any stories could be recorded.
Baker has been involved in the reading of old Target novelizations in the BBC Audio range of talking books, "Doctor Who (Classic Novels)". Doctor Who and the Giant Robot was the first release in the range read by Baker, released on 5 November 2007, followed by Baker reading Doctor Who and the Brain of Morbius (released 4 February 2008), Doctor Who and the Creature from the Pit (released on 7 April 2008) and Doctor Who and the Pyramids of Mars (released 14 August 2008). In October 2009, Baker was interviewed for BBC Radio 4's Last Word to pay tribute to the deceased former Doctor Who producer Barry Letts. He described Letts, who originally cast him in the role, as "the big link in changing my entire life".
He returned to the show in the 50th anniversary special, The Day of the Doctor, in 2013, playing a character who calls himself The Curator. On 20 November 2013, Baker revealed that he would appear, stating "I am in the special. I'm not supposed to tell you that, but I tell you that very willingly and specifically; the BBC told me not to tell anybody but I'm telling you straightaway." The episode saw Baker in the role of a mysterious curator in the National Gallery.