David Lynch’s new film, Mulholland Drive, tells a story that is as old as Hollywood, one the town knows intimately, but tends to leave aside as a subject for movies.

Already, though, time to stop and loop back. There are problems with that opening. The words, “David Lynch’s new film, Mulholland Drive, tells a story…” need work.

For one thing, Lynch’s new film isn’t exactly a pure film at all. It’s partly a glimpse of a television show that no one ever saw. About two-thirds of it was shot early in 1999 as the pilot for an open-ended series for ABC, the network for which Lynch once made Twin Peaks. You can see its TV roots in the delicious languor with which it sets out its stall, taking time to tease loose just the tips of threads that promise to play and tangle into a dark velvet tapestry, stretching, possibly, into eternity.

That never happened because ABC saw it, took fright, and axed the series. It was too slow, they said, and too aberrant. The lead girls (and they play girls, as well as women), Naomi Watts and Laura Elena Harring – aged 32 and 36 – were too old. And what was with the guy at the diner? Was it even a guy? And the pinhead behind glass? And is that a dog turd… ?

There is confusion as to whether this film is even called Mulholland Drive. That was the name of an aborted TV series. This thing seems, rather, to be called Mulholland Dr., that brutal full stop the nub marking where the rest of it was chopped off.

Settling into it, you see what a strange, sly, funny and blackly beautiful show it might have been; at the same time, you wonder how Lynch is ever to resolve all he has begun – all this Vertigo and Sunset Boulevard and Farewell, My Lovely – in the space of a single movie, even one running for two and a half hours.

But he has decided – daringly, but in a way that might look as if he has simply given up – to leave it terminally, beautifully, incomplete, and finish instead with some familiar sleight of hand.

This brings up another problem with the words “David Lynch’s new film”. People wonder where the “new” is, express exasperation that here Lynch, once again, seems to do just the same old stuff. There are motifs and textures familiar from his psychotic movie Lost Highway (1996) as well as Twin Peaks and its film, as well as Blue Velvet (1986), and so on, right back into the incredible dirt shadows of his first feature, Eraserhead (1976).

To wonder, though, whether or not Lynch can do anything different is to have forgotten his last movie, The Straight Story (1999), a slow, hushed, folksy masterpiece of sensual humanism, shot like summer brushing autumn. And it’s to mistake Lynch for a different kind of filmmaker. Lynch comes from painting, and, as much as he can, makes movies like he makes paintings, the difference being that movies cost so much more. All his work – films, paintings, music, cartoons, comic strips, even his website, davidlynch.com – run together, one vast, tonal exploration of personal obsessions. A private work, but open to visitors.

Doubles, illness, decay, blondes and brunettes, sex, mystery, cigarettes, violence, rooms, obsession, telephones, fear, creeping unease; these are some of the things making up Mulholland Dr. – and Eraserhead, Blue Velvet, Wild at Heart (1990), Twin Peaks and Lost Highway. Even the everyday raptures of The Straight Story seem an extension of those remarkable Twin Peaks moments when characters would pause to rhapsodise over hot coffee and fresh air.

Images, textures, gestures repeat across decades. In Mulholland Dr. a gangster (craftily played by Lynch’s mischievous genius composer, Angelo Badalamenti), freezes his face and lets espresso just dribble out his mouth, and it’s suddenly a detail from Lynch’s student piece, Six Figures Getting Sick (1967). A woman’s face in a doorway echoes from Eraserhead to Blue Velvet. Blue Velvet ends with a birdie in a tree, which is how Twin Peaks begins. Curtains, stages, spotlights. That flickering light. Pop music that sugars despair. A low buzz and scrape and clank, like a derelict factory sighing.

Lynch’s fetishistic endeavour has one Hollywood parallel, in the output of that slyer surrealist, Alfred Hitchcock. In some ways, Lynch is the closest to Hitchcock we have, but the differences are vast and crucial. Hitchcock had his movies largely completed in his head before he arrived on set; Lynch is open to chance and accident. Like Hitchcock, he casts certain parts for shapes and faces rather than acting. Unlike Hitchcock, he will bend films to accommodate what actors bring.

(To Mulholland Dr., Naomi Watts brings the performance of a career).

Lynch’s obsessive art really has less to do with other movie-making than with painting; Edward Hopper’s preoccupation with strange corner places at strange corner times of day; or Georgia O’Keeffe’s with her flowers across the decades. O’Keeffe has offered the best explanation: “I said to myself, I’ll paint what I see, what the flower is to me, but I’ll paint it big and they’ll be surprised into taking time to look at it …”

For all that he’s engaged on this private odyssey, however, Lynch understands Hollywood like few other directors. Orson Welles was one; Welles came up with the line, “It’s a bright, guilty world.” Mulholland Dr. is the movie adaptation of that phrase.

Lynch lives near the road his film is named for, a twisting road for a twisted story. Commenced in 1923, the same year the Hollywoodland Development Company erected its sign in the hills, it snakes up into the west off the Hollywood Freeway, at first in voluptuous curves, then degenerating to a dusty squiggle, a palsied artist’s attempt at a straight line, and a shrouded place for illicit liaisons, addiction, and hidden bodies.

“You feel the history of Hollywood in that road,” Lynch has said, and it percolates through his film. Like Twin Peaks, Mulholland Dr. is saturated in place, its haunted, hushed, emptied Los Angeles, with its watchful palm trees, is as much a character as the little town with its ghoul-haunted woodlands was. There are old stunt ranches, where cowboys practised falling, and echoes of Howard Hughes, unspeaking, all-powerful and demented, locked in hygienic darkness and communicating through his private surveillance corps.

Where Twin Peaks had a passing Jane Greer – once the dark-eyed girl who paved Robert Mitchum’s road to ruin in Out of the Past (1949) – Mulholland Dr. has Ann Miller, the tall, unbreakable china doll who once stole On the Town (1949) from Gene Kelly and Frank Sinatra as a tap-dancing anthropologist in green. And, five years before that, stole Louis B Mayer’s heart.

How Lynch ties all this up in the final section of Mulholland Dr. seems at first willfully disorienting: confusing and senseless in a way that nevertheless holds its own pleasure because it seems such an audacious “fuck you” to the audience. However, Mulholland Dr. is one of the few recent films that doesn’t hold its audience in contempt. On the contrary, Lynch honours us, stimulates, takes curiosity and intelligence for granted.

Like any of the killer femmes in the old, black Los Angeles noirs whose ghosts stir here, this seductive movie plays hard to get, confident you’ll return. You can’t get it out of your head. Go with it, go again, and it becomes clear how, far from being unfinished or willfully obscure, this is an astonishingly rigorously structured film, in which everything changes to stand revealed in the light of everything else. It’s the dying dream of Point Blank (1967), retooled for the dream factory itself, polished bright, but getting dimmer and guiltier by the second.

This brings up the other problem with my opening, “David Lynch’s new film, Mulholland Drive, tells a story …” As with any femme fatale, the real story of Mulholland Dr. is not what you thought it was at all, but something smaller, deeper, more desperate.

Its real story is not about Hollywood, but something far older. About one who loved not wisely, but too well. Who, used up, let down and perplexed, reached for revenge and was then assaulted by the knowledge that, more than anyone, the person she betrayed was the person she once was.

This new thing from David Lynch: the most overwhelmingly emotional love story in decades. But here I’ve run out of space. No room to talk about Roy Orbison. At the end, I’m no further than the first sentence. It was over before it even started.

Film review, The Scotsman, December 2001. Republished in Supporting Features.