Can you create API Gateway methods using a CloudFormation template without a Lambda proxy function?

Yes CloudFormation supports creating API Gateway. The first step for customer is familiarizing with different integration options available for an API.

In the previous answer post, swagger integration option was mentioned. Customers can start API development or prototyping by using swagger editor and other API development plugins.

https://swagger.io/ The next step could be to create API endpoint using CloudFormation. API Gateway version 2 is available and it provides lot of benefits. We could start without backend integration of Lambda and use integration type as MOCK.

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-apigatewayv2-integration.html

Once we have the skeleton ready we could create the backend in either AWS Lambda or HTTP endpoints of choice and wire them. This enables independent development of back end and testing based on API contract.