Martin Scorsese’s new Dylan documentary, Rolling Thunder Revue: A Bob Dylan story, a sequel to his 2005 Dylan documentary No Direction Home, which charted Dylan’s early life and career as a folk singer, takes on a series of concerts that Dylan organised over the course of 1975 and 1976. Billed as the Rolling Thunder Revue, Dylan played small venues, with a large supporting cast that included his performing partner in the early sixties Joan Baez, the poet Allan Ginsberg – charged with opening the concerts with readings – and the old-school folk-singer Ramblin’ Jack Elliott, among others.

Scorsese follows the same formula as his first (and brilliant) documentary No Direction Home, in mixing together contemporary footage shot at the time (taken from Donn Pennebaker’s 1967 documentary Don’t Look Back, shot in jumpy cinéma vérité-style), excerpts from the four-hour dreamscape film Renaldo and Clara (conceived by Dylan himself and shot by filmmaker Howard Alk and playwright Sam Shepherd during the tour), together with footage of the musicians in their off-time, filtered through a weird, purple solution, and bright, intense close-up shots of performers in concert, all cut with straight interviews with the cast and Dylan.

But nothing is quite as it seems. The film opens with Dylan trying (and failing) to tell the viewer what the Rolling Thunder Revue was:

“I don’t have a clue because it’s about nothing. It’s just something that happened 40 years ago… I don’t remember a thing about Rolling Thunder. It happened so long ago I wasn’t even born. So what do you want to know?”

A familiar sentiment that tacks onto one of Dylan’s most popular formulations “He not busy being born is busy dying” from “It’s alright Ma I’m only bleeding”, it was quoted by Jimmy Carter in his acceptance speech at the 1976 Democratic National Convention and Al Gore on Oprah during his ill-fated 2001 Presidential run. Like “How many roads must a man walk down?”, it’s one of those readily quotable lines that keeps Dylan resonant – a quotability that chimes readily with the glib, popular cliché of Dylan as a prophetic “voice of a generation” protest singer and poet all in one (in reality, he only wrote two truly great protest songs: “The Lonesome Death of Hattie Carroll” and “Only Pawn in Their Game”, neither of which offers simple moral lessons).

With a wry sideways look at the camera, he continues: “Life is about creating yourself. Life is about creating things.”

In 2002, Dylan decided to introduce every live show he did with this announcement: “Ladies and gentleman please welcome the poet laureate of rock-n-roll. The voice of the promise of the 60s counter-culture. The guy who forced folk into bed with rock. Who donned makeup in the 70s and disappeared into a haze of substance abuse…” etc etc. and I wonder whether Dylan’s rather glib tone, at his most half-arsed, brandishing some WikiQuotes-inspired pop-existentialism, is just another piss-take of the Bob Dylan persona. Scorsese, the interviewer, doesn’t press him – for him, there it is, this really is Dylan’s truth at work – a little indirect, a little mysterious, hard to pin down.

Dylan’s persona has indeed gone through a series of transformations that sometimes matched the spirit of the time, sometimes jarred with it. As a teenager, he played gigs in his Midwest hometown, Duluth, Minnesota as “Bobby Vee”, a popstar who had then had a couple of minor hits with a mellifluous bubblegum voice, then left for Greenwich Village to join the folk scene there, to become a “misty-eyed” balladeer, with a rasping timbre straight from the harsh world of the earliest folk traditions of the Appalachian Mountains (see Dillard Chandler as a good 20th century comparison).

Then on to rock phenomenon status, captured in Donn Pennebaker’s documentary film Don’t Look Back, filmed over 1965 and 1966 and released in 1967, cruelly leaving his old singing companion Joan Baez off stage, then in one scene mocking the singer Donovan – Dylan watches on as he plays “To Sing For You”, a rather trite romance (“When the night has left you cold and feeling sad, / I will show you that it cannot be so bad” etc), then he picks up the guitar, grins and launches into “It’s All Over Now, Baby Blue”, (“The Vagabond who’s rapping at your door / Is standing in the clothes that you once wore”, at which point Donovan appears visibly to shrink into himself).

Much of the rest of the documentary trades off the notion that Dylan is a kind of trickster – Scorsese introduces Van Dorp, a figment of Scorsese’s pop-Dylan-esque imagination, a fictive parallel with the filmmaker Howard Alk, who shot Renaldo and Clara: “I’m the one who made this. You’re using it. This wouldn’t exist without me. I’m the filmmaker here”, Van Dorp tells Scorsese.

It is “Mr. Tambourine Man” that plays out across Scorsese’s opening montage, a clunky series of segues between President Nixon sonorously intoning “Let us set for our goal in 1976 to move forward in the realm of the American spirit…”and a weirdly costumed Uncle Sam singing “The Star-Spangled Banner” before a slightly bemused crowd (1976 was the United States Bicentennial).

Cue the thought – the present is no place to be. Time to follow Dylan, away, away out to his “circus sands” (indeed, the musicians of the Rolling Thunder Revue are given special circus names in the final credits). In setting up these associations, Scorsese effectively imposes a pattern that chimes with a partial appreciation of the Dylan-esque – and somehow also misses the point that, for all his “mad, mystic hammering”, Dylan is a very precise kind of artist, for whom the lyrical themes of wild, ecstatic possibility – the evanescent image of a freer world achieved through a conscious stripping back of the accoutrements of the self (“I is another”, as Rimbaud put it) – are balanced off by a keen awareness of the limits to human experience: “death’s honesty”, as he sang once.

Listen to “Mr. Tambourine Man” from that extraordinary B-side of Dylan’s fifth studio album Bringing it all back home, with its chorus, “There is no place I’m going to”, and disciplined negative formulations, “I’m ready for to fade”, “play a song for me”, “my senses have been stripped”. The future is blocked off, with “evening’s empire” just around the corner. Then the tight, twisted patterns give way to an ecstatic series of images, pure Rimbaudian excess, with Dylan left with “one hand waving free”, against a “diamond sky”.

Even then, the song returns back in on itself, “sands” and “hand” bring us all the way back to “evening’s empire… returned into sand” in verse one. And after it rounds off, “let me forget about today until tomorrow”, Dylan’s quixotic scrambling of eschatological time (Matthew 6:34: “Take therefore no thought for the morrow”), is followed straight away by the final chorus, a repeat of the plaintive, “there is no place I’m going to”.

The picture of Time that a song like “Mr. Tambourine Man” discloses is very different from the futurity of, for example, “Like a Rolling Stone”: although its lyrics are all snarl (“As you stare into the vacuum of his eyes”), they’re set to an infectious tune, with that bounding guitar and organ thrum that just seems to skip along. We are borne out from the present into a bright, American future, a young country where History is no place, criss-crossed by bright highways: “no direction home”, indeed: “When you got nothing, you got nothing to lose.”

Or take the way time unfolds in Dylan’s country-style album Nashville Skyline, which opens with an infinitely slowed-up cover of his troubadour era hit “Girl from the North Country”, with Dylan and Jonny Cash on vocals, each line dragged out as far as it can go, while still resembling the taut original. It’s a very odd song, just about maintaining its shape without self-pity nauseatingly permeating the whole thing. Another track, “To Be Alone With You”, kicks off with a halting instrumental section that only gathers pace after the musician chimes in: “Is it rolling, Bob?” Just about, just about, you think.

For Dylan, women always steal the show, of course. Remember the “Sad-Eyed Lady of the Lowlands”, with her “mercury mouth”, or the “woman” of “Just Like a Woman”, with “her amphetamine and her pearls”, or “those visions of Johanna that conquer my mind”. There’s Patti Smith appearing early on in Rolling Thunder, all frenetic bright-eyed intensity, with Dylan observing in the audience. In the tour performances, Dylan adopts her gyrating style along with a punkier, souped-up sound – not so far away from the driving surf beats of Smith’s early material.

“They say ev’rything can be replaced / Yet ev’ry distance is not near”, Joan Baez and Dylan sing together: “I see my light come shining / From the west unto the east / Any day now, any day now / I shall be released”, in a cover of “I shall be released” from The Basement Tapes.

Baez and Dylan singing together has always moved me close to tears, a common thing for Dylan fans. Will Self, who burst into tears on viewing footage of them playing together at the Monterey Jazz Festival in 1963, described himself as “overcome by the ethereal beauty of this juvenescent couple – and their bright, shining talent”. A theme echoed by Dylan in Rolling Thunder: “Joan Baez and me could sing together in our sleep.” In their onscreen magnetism, we feel the promise of some resolution to Dylan’s recurrent themes of isolation and loss. So many of Dylan’s songs gain shape in an address to a twinned feminine “You”, “Mama, You Been On My Mind,” for example, or indeed, “Girl of the North Country”, a reverie on sending a letter to someone far away: “Remember me to one who lives there / She was once a true love of mine.”

Every distance might not be near, it is true, but just for a second, it doesn’t feel like that. It is here that we feel closest to the authentically Dylanesque – the manufacturing, as Michael Neve put it, of a “twinned-self… one that has journeyed thousands of miles, walked on dozens of stages, changed shape, loved and lost”.

T.S. Eliot wrote in his Four Quartets, on ageing and personal biography, that he feels conscious of “not the lifetime of one man only / But of old stones that cannot be deciphered”. And I think this is a maxim that Scorsese could have heeded, in that this attempt to author a definitive portrait of Dylan in the 1970s somehow leaves the viewer with an overwhelming sense that Scorsese is treading over old stones that he cannot decipher.

There is a gentler interpretation, of course: that this is true of all who love Bob Dylan, that we have our own hang-ups and missteps, and that we know very little, in the end, of why his music moves us so often to tears, to frank astonishment, indeed to awe.