In a letter from Fred Salerno to Redstone, made public by the company, Salerno also defended Viacom management's handling of the possible sale of a stake in movie studio Paramount.
"Sumner, they tell us they won't make you available for fear that such a meeting would become the source of more litigation," Salerno wrote. "In reality, putting up a wall around you ensures more litigation – and that is not what we want."
A representative for Redstone declined to comment.
Last month Redstone ejected Viacom Chief Executive Philippe Dauman and another director from the board of National Amusements Inc, which controls 80 percent of the voting shares of Viacom.
Dauman filed a legal challenge to that maneuver, arguing that Redstone was being manipulated by his daughter Shari.
Salerno and Viacom's other independent directors then vowed to fight any attempt to oust them from the board, saying they found "inexplicable" the assertion that Redstone was mentally competent.
The sale of an interest in Paramount has been an issue of contention in the battle for control of 93-year-old Redstone's $40 billion media empire, which includes Viacom and CBS Corp (CBS.N).
Redstone said in a statement last month he believed that keeping 100 percent of the film unit was in the best interest of Viacom but could be swayed by a plan that convinced him otherwise.
In the letter, Salerno recounted a meeting in February in which Dauman told Redstone about new opportunities for Paramount.
"He reported that you didn't react, other than to nod when he asked you if you heard him and understood what he said to you," Salerno wrote.
At a subsequent board meeting, Redstone did not speak up when the board discussed Paramount, Salerno said.
"We are quite concerned that your voice – and views – are not being heard," Salerno wrote. "When your phone is dialed into our board calls from your home, no one says a word. When we ask for your vote, all we hear is silence."