As the healthcare company battles reputational issues, investors are targeting its directors.

By Mina Kimes, writer

FORTUNE -- Every year, shareholders at public companies vote on whether their board members deserve another term or not. It's typically a non-event; the average director of an S&P 500 company wins 96% of the vote, according to proxy advisory firm ISS. But the directors of healthcare giant Johnson & Johnson, which hosts its annual shareholders' meeting Thursday, may face a rockier path to reelection. Investors have begun to turn against the board, with a growing contingent of shareholders voting against J&J's nominees.

In 2011, the average J&J (JNJ) director won approval from just 88% of shareholders, down from 94% in 2009. That may not sound like a steep decline, but it's actually quite low for a large corporation, according to Charles Elson, director of the John L. Weinberg Center for Corporate Governance at the University of Delaware. "When you get into the 80s, that's a real problem," he says.

Two J&J directors -- Charles Prince, the former CEO of Citigroup (C), and Michael Johns, the chancellor of Emory University -- received support from about 80% of voters, which is well inside the "danger zone," according to Paul Hodgson, a senior research associate at GMI, a corporate governance ratings firm. "That's a very substantial proportion of shareholders," he says. "It should be enough to make a board sit up and take notice."

MORE: Wal-Mart's board: Can they handle the Mexican heat?

A spokesperson for J&J wrote in an email that the board will analyze voting results after its annual meeting, as it does every year.

Red flags: Disappointing financials, recalls

It's easy to see why J&J's shareholders are frustrated. In recent years, the company has been charged with several legal violations and endured a seemingly endless series of product recalls. Its reputation has taken a hit. Back in 2009, before the recalls began, J&J was ranked fifth on Fortune's World's Most Admired Companies list. In 2012, J&J's overall rank dropped to 12.

The company's financial results have also been underwhelming. J&J's stock has delivered a total return of 5.3% over the last two years, while the S&P 500 has returned 17.5%. Its net income declined in both 2009 and 2011, in part because the company lost more than $1 billion in sales because of recalls of over the counter drugs. More