A growing number of banks and retailers are moving beyond Social Security numbers to verify your identity. They're relying on such personal details as your car color, your father-in-law's name and the city you lived in five years ago.

No, you never gave them this information; rather they pulled it from public and private databases. These private details are increasingly being used to approve you for credit, give you access to your account online or to verify that you — rather than an impostor — is making a purchase.

It's the latest effort by financial institutions to fight a growing threat of identity theft from online "phishing" and other scams. Chase, HSBC, Vanguard, American Express and Barclaycard US use this customer-verification technique. In the past two years, the technology has been adopted by six of the top 10 U.S. banks and thrifts, says Verid, a provider of the technology.

"Names, addresses and Social Security numbers are no longer the unique identifiers they used to be," says Kevin Watson, Verid's CEO. "You can phish someone's password and Social Security number, but you can't phish someone's memories."

Privacy advocates worry that the technology could violate consumer privacy. "This is pretty scary," says Paul Stephens of the Privacy Rights Clearinghouse. "They're taking information from public records, but the individual can't control how this is used."

If you use your credit card more times in one day than usual, your bank may call and ask about the make of your last car before approving a transaction. You might also be quizzed on whether John is the name of your father, brother or brother-in-law.