You can only embed images directly in New Sites. To insert documents you can upload them to Google Drive, share them publicly* and insert them using the From Drive option in the INSERT menu.
*You can also publish the document in order to embed it into your site. See this information from a docs help file:
To make a document, spreadsheet, or presentation, available for a large audience to see, publish a file. After you publish your file, you’ll have a URL that you can send to anyone or embed into your website.