Girl walks into a bar. Says to the bartender, "Give me a Diet Coke and a clear sight line to those guys drinking Miller Lite in the corner."

No joke. The "girl" is Emma Gilding, corporate ethnographer at Ogilvy & Mather, one of the world's top advertising agencies. Her assignment is to hang out in bars across the country, watching guys knock back beers with their friends. And wipe those smirks off your faces. This is research.

As directors of OgilvyDiscovery, Gilding and her partner, Johanna Shapira, supervise a team of researchers who follow consumers for hours or days at a time, filming them in their native habitats. Their goal: to capture the telling moments that reveal what consumers actually do with products, rather than what they say they do.

That might mean catching a heart-disease patient scarfing down a meatball sub and a cream soup while extolling the virtues of healthy eating, or a diabetic vigorously salting his sausage and eggs after explaining how he refuses jelly for his toast. The subjects could be Mexican middle-schoolers, goofily acting their age–and then sagaciously commenting on how their time for tomfoolery is running short. Gilding and Shapira have seen and heard all of that.

Since at least the mid-1990s, the advertising industry has been fighting a war on multiple fronts. Media fragmentation, declining network audiences, information clutter, and ad-zapping technologies have made agencies increasingly eager to find an edge in reaching consumers. Some larger firms believe that ethnographic research such as Gilding and Shapira's can help identify consumers' emotional hot buttons, allowing them to craft messages with more resonance. As Bill Gray, president of Ogilvy & Mather, New York, says, it's one more tool to help "tip the balance of the environment in favor of our brand" in a marketplace where consumers' engagement with a brand can mean the difference between a best-seller and an also-ran.

It may sound a little like what we've come to expect from reality TV. But don't confuse ethnography with The Osbournes. "This is an authentic experience, not one that's produced," says Gilding, the 35-year-old British native who launched OgilvyDiscovery five years ago. "Our job is to bring a whole world more realistically to life." Among others, OgilvyDiscovery has plumbed the worlds of asthma sufferers for Bristol-Myers Squibb, small-business owners for American Express, and damp babies for Kimberley-Clark's Huggies.

The methodology goes something like this: Gilding and her team put out a call through recruiting agencies for potential subjects. They rigorously screen candidates against a set of qualifications, typically rejecting 50% or more until they get the right people. Subjects are paid, but that should not be the point: Gilding wants experienced consumers who are brand experts. "In most cases, they know more about the brand that we're looking at than the CEO of the company."

Once chosen, subjects agree to allow videographers to follow them for a day or longer, documenting the minutiae of everyday life and gathering information on everything from emotional engagement with a product to environmental cues on its place in the home and the psyche of the user. After an initial awkwardness, the camera typically seems to fade away, allowing an extraordinary candor. "There's something really incredible about someone asking you to tell your story for eight hours," says Shapira. "It can almost become a confessional experience for people."

It can also make for some long, long nights. Take that evening at the bar. The OgilvyDiscovery crew was trying to decode the male-bonding rituals of twentysomething Miller Lite drinkers. As a videographer filmed the action, Gilding kept tabs on how closely the guys stood to one another. She saw that high-fiving was out, fist-pounding was in. She watched as guys gathered, dispersed, then reconvened around the call for "Beer!" She noted how hierarchies were established, and conflicts resolved. She eavesdropped on stories, and observed how the mantle was passed from one speaker to another, as in a tribe around a campfire. And she didn't drink beer.

Back at the office, a team of trained anthropologists and psychologists pored over more than 70 hours of footage from five similar nights in bars from San Diego to Philadelphia, extracting 20 minutes that seemed to capture the essence of the Miller Lite drinker's experience. One key insight: Miller is favored by groups of drinkers, while its main competitor, Bud Lite, is a beer that sells to individuals. That insight reinforced research that Tom Bick, Miller's senior brand manager, had been conducting independently. "We could see from our own demographics and focus groups that our drinker base was somewhat different than Bud's," he says. "But we couldn't put our finger on it."

Ogilvy's research confirmed that the archetypal Miller drinker felt more comfortable expressing affection for friends than did the Bud Lite boys. "We felt the Bud guys were much more about impressing each other," Bick says. "The problem was, how do we convey that without completely going off the edge? We didn't want a campaign that got all sanctimonious. It's a beer ad."

Enter the Ogilvy creative team, which riffed on their own experience in bars. The result was a hilarious series of ads that cut from a Miller Lite drinker's weird experiences in the world–getting caught in the subway taking money from a blind musician's guitar case, or hitching a ride in the desert with a deranged trucker–to shots of him regaling friends with tales over a brew.

Steve Hayden, Ogilvy's vice chairman and worldwide creative director, said the real-life video helped the team get the details just right. "In advertising, one of the hardest things to do is to re-create a world, from behavior to real human cadences and tones. The tape showed us, for example, that these guys don't speak in neat sentences or well-outlined paragraphs. It let us bring a level of verisimilitude to the execution that was just terrific."

That's just what Gray had in mind when he hired Gilding back in 1999. At the time, he was looking for a way to beef up the agency's strategic capabilities and sharpen its creative output. Gilding, whose background was in cultural analysis, was consulting on marketing and producing documentary films in Leeds. She cold-called Tro Piliguian, Ogilvy's North America CEO, and managed to talk her way into a meeting in New York. The result? A job offer, and an edit room that was in a closet that still reeked of the art department's adhesive spray.

In 2000, Gilding was joined by Shapira, 38, a Canadian strategy specialist who had worked for J. Walter Thompson in Bangkok. Her interest in ethnography was piqued while researching shampoo among village women. The client thought the product's natural ingredients would be its selling point. But villagers told Shapira they wanted glitz. Science! Plastic! "I learned that if you really want to know what's going on around you, you just have to shut up and listen," she says.

The two, who finish each other's sentences and are partial to ad agency black, admit their relationship can be contentious. "We think very differently, so we often battle things out from different points of view," Gilding says. "But ultimately, we have the same relationship to the data, which is one of ethical responsibility." Their team now includes six full-time producers and researchers, and a phalanx of consulting anthropologists and psychologists, who are matched with clients by specialty. The group has moved from its original closet to a room at Ogilvy's Manhattan headquarters filled with computers and video equipment.

Ogilvy isn't the only company to discover the methodology. Young & Rubicam, Saatchi & Saatchi, J. Walter Thompson, TBWA\Chiat\Day, and a raft of independent consultants all have used ethnographic research, as have companies ranging from Microsoft to MTV. In fact, Gilding's sort of analysis has been in and out of favor among marketing researchers since the 1930s, according to Kellogg School of Management marketing professor John Sherry. Most recently, such methods had been shunted aside in favor of more quantitative techniques. But now, Sherry believes, they've regained credence as advertisers have become more interested in brands' affective qualities. "Most marketers have traditionally had an engineering or product-oriented perspective," he says. "But with all the talk about the 'experience economy,' they're starting to pay attention, because here's a methodology that prizes it."

Interest in this technique has also grown as frustration mounts with the limitations of more conventional research tools like focus groups and surveys. "A focus group is a like a chain saw," says Americus Reed II, a marketing professor at the Wharton School of Business. "If you know what you're doing, it's very useful and effective. If you don't, you could lose a limb." Group opinion can be swayed by more vocal and strident members, he says, and findings can be skewed by participants' reluctance to share their true feelings in front of a group.

Ethnographic research overcomes some of those limitations by its duration and sometimes extraordinary intimacy. Shapira cites one study for Depend incontinence products that focused on women. On the tape, a woman in her bedroom confesses to the videographer that she's never asked her husband how he feels about her condition. "Maybe I don't want to hear the answer to that," she says.

"We're not talking about the fineries of marketing here," says Shapira. "These women cannot get help because they cannot speak." The tape also helped the creative team get over its squeamishness about the subject. Shapira tells of one copywriter who confessed he was mortified to land on the account. "I was so bummed out," he said. "But now I have never felt more passionate about anything in my life. These women just reach out and grab you by the throat and make you want to make the world okay for them." Depend ads are still in testing.

Ethnographic research is not a panacea. For one thing, it's expensive. A typical study includes 6 to 10 subjects, each paid as much as $500–and that's a fraction of the total tab. The process is time-consuming: Start to finish on an average-sized study customarily takes Ogilvy's group four to six weeks. Even then, the results don't provide clear guidance to a client accustomed to more quantitative methods. "You won't come away from a Discovery film saying, 'Okay, then the marketing plan for next year should include a 30% buy on billboards,' " Shapira says.

Paco Underhill, whose books Why We Buy and Call of the Mall are classics of modern retail ethnography, confesses to a bigger concern: How does this research translate into sales? "Advertising is some of the finest visual communication our culture generates," he says. "But ethnographic research is still at some distance from where rubber meets the road in terms of sales."

Bick concedes the point. The Miller Lite ads, for example, got high marks from audiences for their entertainment value and emotional resonance. But they weren't as successful in driving sales. "What people got from that campaign was that it was about time with your best friends. The emotional side was very powerful. Its downfall was that it didn't work on a more fundamental level." That is, it didn't connect the emotional appeal with a compelling enough product message.

But Bick isn't complaining. Indeed, he thinks it may be time to deploy another team to revalidate the initial findings and discover what's new. In fact, he's considering similar research for Miller Genuine Draft, Miller Lite's sister brand. "So much other research is done in isolation of social groups," he says. "But brands are adopted by a tribe of users." Ethnographic research "helped us to understand the Miller Lite drinker and his friends as genuine people." Now, he just needs to figure out how to sell those guys more beer. nFC

Sidebar: These Aren't Home Movies

Ethnographic research is more than just a guy with a video camera channeling reality-TV guru Mark Burnett. Here's what you need to know before you hire a researcher.

Check the bona fides. Make sure the researchers you hire have graduate school credentials in anthropology or sociology. This is a specialized discipline that goes beyond traditional market research.

Allow enough time–and money. Recruiting appropriate subjects, filming, and analyzing data isn't an overnight project. A typical study takes four to six weeks to do properly, and costs in the neighborhood of $50,000 to $100,000.

Match the technique to the problem. For many problems, focus groups or surveys work just fine. Call in the ethnographers when you need a more detailed understanding of how consumers use your product.

Fast Company senior writer Linda Tischler drinks Zinfandel.