The difference is right there in the name: a mortgage lender lends the money to the borrower. A broker, by contrast, connects the borrower to a lender from their existing list of partners, acting as a middleman, and builds in a fee for their services. Lenders are also held to stricter standards in order to operate; lenders have to maintain a minimum net worth, so they always have the ability to back any transaction they lend on. Additionally, they need to maintain good standing with their investors.
By coming directly to a lender, you get the benefit of reliability, and you save money by cutting the middle-man out of the equation.