Western Asset Management Co. plans to start buying non-agency mortgages the way a gentleman orders a bespoke suit.

The bond firm, with $469 billion in assets, will take on the role typically played by Wall Street banks, specifying to lenders the mortgages it wants them to make—from interest-only to those with higher debt-to-income ratios—then buying them. Other firms such as Caliber Home Loans Inc. are also seeking to capitalize on demand for loans that don’t meet requirements for taxpayer-backed programs or qualified mortgages, known as QM loans.