trickyform

Node.js File Uploads Course

Banging head to desk to get file upload working from within Node?

Two hours spent reading yet another tutorial, but files still get created with 0 bytes in your project

You know how you can spend two hours reading a tutorial and realize it doesn't leave you fully off on your own. Files still get created with 0 bytes in your project.

There's a lot of information out there, and it's not easy to decide which pieces are the best use of your time. Examples scope out features important to you. When you don't know all the pieces of the puzzle, transferring code snippet to your exact use case is difficult.

Choosing a library is a time suck

Search around npmjs.com gives several convincing candidates for processing file uploads. Which library is the right one for you? A sharp developer realizes it depends on the situation. What factors should you consider when making the decision?

The obstacle between you and getting work done is to read through the documentation of five different libraries. And that takes time and effort.

Attempt to collect fields along with file ends up with undefined values

A form with a lone file upload input is rarely a real-world scenario. A user registration form with a profile picture is nice, but it needs a username and email address to be useful. You need to collect fields along with files.

Each library has its own way of passing input fields. If you don't pay close attention, you end up with values that are undefined. The frustration... and time wasted.

Leaving uploaded files on hard disk can be used as an attack vector

Have you prevented files from being left around in the file system? Get enough of them, and they will fill up your server. It's child's play for an attacker to fire several uploads. They'll fill up your hard drive and stop the server from serving real users.

It's one thing to hack up a quick proof-of-concept file upload that's not exposed to the outer world. But in order to really sleep well at night, you want to be sure no servers can crash due to file uploads.

How to give a link back to a file - one that doesn't result in 404

Uploading files is one thing, but what about accessing the uploaded files. You want to send back a link the client can use - one that doesn't result in 404? What good does an upload do if you can't view the files? So you need a logic that serves wherever the files are stored through your application to the end-user.

Get your files, safe and sound, uploaded on a server

What if you knew precisely how to get your files, safe and sound, uploaded on a server. You'd know all the pieces of the puzzle from reading the HTTP request all the way to pushing the files to their final resting place. You can actually get work done.

Leverage existing packages, and use your time to implement new features

By using existing libraries, you can cut development time. When you pick the right libraries that match your use case, the freed time can be used to implement new features. You'll get to see the finish line quicker.

Validated fields at your fingertips with full values: email, username, shoe size

Make your form processing collect both fields and files. Go even one step further, and validate the collected fields. This way, numbers are numbers just as you expect them. And, why don't you trim any extra whitespace around inputs while you're at it.

Leave office knowing your servers will stay up

Offload files to a dedicated file storage server. This way, Node.js servers will keep running and responding to user requests even while receiving large amounts of files. Leave the office knowing your implementation is solid and won't be the source for monitoring alerts.

Users can access their files, for your backend serves frontend needs

Give users access to their files. Provide the frontend developers a clear API they can use to build a tailored UI for swiping through uploaded files. Your backend serves frontend needs.

Video course where you'll learn Node.js file uploads

Get Node.js File Uploads course, and you'll learn how to read file upload requests, make sure files get to their destination, collect fields, and serve the uploaded files to users.

In Node.js File Uploads course, you'll learn

  • REST API that suits file uploads
  • backend form processing
  • busboy library for parsing file upload request
  • read text fields along with files
  • use streams for efficient file handling
  • reject invalid submit data
  • choose between database and cloud storage for file storage
  • clean up files left behind when upload fails

You'll learn from conceptual lessons that hone in the right mental model and code screencasts where you'll roll up the sleeves and apply them.

Build a backend for a real project from the ground up

There's no better way to learn than to open the editor and code. And while you do this in the context of a real project, the concepts will sink in better. You'll get to see all the parts that are needed to make file uploads happen.

In this course, you'll build the backend for a Product Catalog app. And you'll do this from the ground up.

Product Catalog manages the inventory of an online store. You can add products with metadata such as title and price and attach any number of images that showcase the product.

This is where the file upload part comes into play. We have to write code that accepts a number of images, and that also serves them to the user once they're uploaded.

You'll be given a fully operational pre-made React frontend. We'll start the backend implementation by defining the APIs from UI mockups and writing a dummy backend that responds with hard-coded values. The dummy backend allows us to test-drive the frontend.

We'll then implement the backend one endpoint at a time. You'll get to see all the sides that are needed to make uploading and serving files to users happen.

Take a look at the finished app:

Product Catalog project
See what we'll build

Move files efficiently using streams

A production-grade system wastes as little resources as possible and is able to perform even under heavier loads. You'll learn to handle moving file data efficiently using streams. No piece of data is held longer than what's necessary, and every chunk received is pushed forward as soon as possible.

Learn to offload files out of Node.js server to external storage

Files are best offloaded out of Node.js server and onto external storage. You'll learn to choose between cloud storage such as AWS S3 and Database binary objects. You'll also write detailed implementation based on PostgreSQL BYTEA binary objects.

Master handling errors, and not just the happy path

You'll not only learn the happy cases but also how to deal with internal errors. If a request times out or processing ends in a system error, it's important to stop processing the file upload request, close any opened resources, and bail out early.

What's included

You'll get 27 video lessons and 15 text lessons:

Choose Right Way to Upload Files
thumbnail
Understand How Browser Submits Form
thumbnail
Upload File Contents With Multipart Form Data Encoding
Handle Files Efficiently With Streams
thumbnail
Avoid Storing Intermediate Files
thumbnail
Stream Uploaded File to File Server
Choose Right npm Package to Process File Uploads
thumbnail
Choose Right npm Package to Process File Uploads
thumbnail
Use Busboy Library to Process File Upload Request
thumbnail
Product Catalog Project
Design REST API That Suits File Uploads
thumbnail
Design REST API That Suits File Uploads
thumbnail
Write Dummy Endpoints in Express.js
thumbnail
Use Pre-Made React Frontend
Choose Between Cloud Storage and Database for Storing Files
thumbnail
Choose Between Cloud Storage and Database for Storing Files
Store Uploaded Files In Database
thumbnail
Organize Project Using 3-Layer Architecture
thumbnail
Implement Application Logic for Create New Product
thumbnail
Connect to Product Database
thumbnail
Design Database Schema for File Uploads
thumbnail
Set Up PostgreSQL With Docker
thumbnail
Insert Product in Database
thumbnail
Organize Database Code with Classes
thumbnail
Use Connection Pool
thumbnail
Test Creating Product Without Images
thumbnail
Use COPY FROM Query to Stream File to Database
thumbnail
Await for Stream Completion With Pipeline
thumbnail
Allocate Image Identifier
thumbnail
Link Uploaded Files and Product
thumbnail
Test Creating Product With Images
Serve Stored Files to Client
thumbnail
Implement Get Individual Product
thumbnail
Implement Get List of All Products
thumbnail
Implement Route Handler for Get Product Image
thumbnail
Implement Application Logic for Get Product Image
thumbnail
Use COPY TO BINARY Query to Stream File From Database
thumbnail
Process Binary Reader Events
thumbnail
Handle Case Where Zero Rows Are Found
thumbnail
Test Viewing Product Image
Validate Incoming Submit Data
thumbnail
Validate URL Parameters
thumbnail
Validate Headers
thumbnail
Validate Multipart Form Data Submit Data
Clean Up After System Errors
thumbnail
Handle Multiple Rows From COPY TO BINARY Query
thumbnail
Stop Busboy From Processing More Request
thumbnail
Clean Up Dangling Files
thumbnail
Handle Stream Errors
thumbnail
Handle User Aborting Upload Mid-Way
thumbnail
Time Out Hanging Connections

About the author

pspi
Hi, I'm Panu! I've been a professional software developer for 15 years. I like to dig deep into difficult concepts and formulate them into logical, easily consumable and visually guided presentations. While I'm not working on my articles or videos, I'm developing software as a freelance developer at my company Cover IT.

Q & A

What library do you use for processing file uploads?

The course uses Busboy. Alternatives such as Multer and Formidable use intermediate files either on disk or in memory. We think a production-grade system is better off storing the files outside of the Node server on a third party storage. It can be a cloud storage provider like AWS S3 or Database that supports binary objects. Busboy uses streams and suits this purpose perfectly.