This is a subjective line which is a matter of opinion. I had to buy some books on this matter, one which I found valuable was

The Copyright Permission and Libel Handbook
A Step-by-Step guide for Writers, Editors, and Publishers

by Lloyd Jassin

Must read for any author.

To answer your question, anything you write about someone is litigable unless they are a public official ... the book tries to explain the gray area. Bottomline, anything can be challenged, what you say is fact, they might say is a blurring of fact. Truth is always a defense to libel but that truth can be questioned.