The U.S. economy grew at an annual rate of 3.2 percent in the fourth quarter of 2013, down from the revised rate of 4.1 percent in the third quarter, the Commerce Department said on Thursday.