Fighting CORS in the flesh — Sequel to a Battle with Browser!

15 min readMar 22, 2024

Not just a “Yet Another Writeup on CORS!”

CORS isn’t as simple in real-life as it sounds theoretically. Negotiating against its behaviour devised by your browser and navigating its errors the right way is a learning experience in itself.

A few weeks back I went through quite a hustle of dealing with CORS while working on a project and that made me explore a bunch of issues and stuff which happens behind-the-scenes with the browser and CORS.

Writing this blog for you to learn from “my” mistakes and gain the associated wisdom.

Man! AI got pretty dramatic with generating such banners xD

Background

This blog is actually the second/last part to a series of two-articles I planned to publish.

The first part was just meant to teach about what is CORS and how it works in the most digestible manner possible.

Feel free to check it out here.
Feel free to skip it if you know how CORS work.

So, let’s dive in!

Context

I am writing a backend for a simple prototype of a P2P loan lending/borrowing website

Language: Go
Framework to write the server: Gin
ORM to talk to the DB: GORM
Database: Postgres

The idea is pretty simple:

Users signup and login.
They either post proposals to offer loan (lenders)
“Hey, I would like to offer a loan upto Rs. 1 Lakh at an interest rate of 6% over a period of 6 months”
Or, they ask for loans (borrowers)
“Hey, I want a loan of Rs. 70,000 at an interest rate around 5% and I can pay it back in 4 months. Is there any loan proposal offering something like that?”

As a borrower, you would want to list the existing proposals on our website to see which one suits the best to you. Simple enough?

So, across this article, I will just talk about a simple endpoint GET /v1/loan/proposals/

This endpoint is used by a borrower to get a list of all the loan proposals the backend is aware of.

Nothing complicated: Just a very rough endpoint with no pagination or filter and just a blind list-operation returning a list of loan proposals.

Current state of data and loan proposals

Let’s start with a situation where the backend (database) has two loan proposal records.

Screenshotting the loan_proposals table in my DB. Look at the two rows

Cool, let’s begin the real jazz

I developed the API and tested it out locally

The story begins from when I implemented the above API and tested a local API testing client like POSTman to test out the API.

All working well, right? Let’s push the code.

Although soon enough after pushing the code, I got a message from my friend working on frontend, saying that the API ain’t working and he’s getting an error.

I was like:

Considering that my local API testing client (like POSTman) could successfully fetch the responses, this meant that it was the browser which was having a problem doing so.

So, I planned to call the API again but from the browser itself.

Did this by opening my browser’s new tab and heading to the Chrome Dev tools.

As I fired the request from there, I saw the following issue!

Dang! I knew this error. And you’d know this too if you have been through the first part of this blog-series. It’s the infamous CORS issue.

Basically, the browser “thinks” that the responses from my backend are not allowed to be looked at.
Somehow the backend has to convey to the browser “Hey, allow this response to be viewed on the browser.”
So, how do you do that? By attaching certain Response Headers in the responses.

The fix was simple:
I just needed to update the backend to set Access-Control-Allow-Origin: * to all of the responses’ headers it sent.

By this way, the backend tells the browser to allow anyone (*) to accept and read the response.

So, I went to my backend’s code and attached a middleware to the root of my routing engine to accomplish the above thing.

In simple words, I attached a function (middleware) to my backend which runs everytime before sending the response. That function just attached the header mentioned to all the responses before they were fired back to the client/browser.

`ctx` variable holds everything about your request and response. See how I am accessing the “response” section of it and updating its headers.

Let’s test out stuff on the local API testing client

Note the second line after “Show Request”. The Access-Control-Allow-Origin header is being correctly set.

All looks good! Note how the third line contained the header coming from the backend.

Time to test the same thing on the browser

Hmm, this was different from the previous error but still something CORS-related.

Seemed like the browser expects an (200) OK status from a pre-flight request it usually makes.

But what the heck is a “preflight request”?
After some digging, I found out that the browser makes a preflight request before making the actual request GET /v1/loan/proposals/
The preflight requests look like this: OPTIONS /v1/loan/proposals/

Damn, I didn’t tell/program my backend about any OPTIONS requests.
Due to this, it must’ve been responding with a 404 Not Found whenever receiving such a pre-flight request, while the browser was expecting an OK.

But why the heck does this pre-flight request exist? Why does the browser like to make such a request?!
It is fired by the browser before the actual request to see which methods, headers, origins, etc. the backend allows.
Fix: The backend upon receiving any OPTIONS request should respond with an OK status straight away assuming that to be a pre-flight request. To respect CORS policies, it should still attach rightful CORS headers to the OK response.

Just return a 204 when seeing a pre-flight request (OPTIONS)

204 is just an OK response with No content.

Let’s test it on the browser again

Aight’ another error!

It seemed like the browser had a problem with the “X-Access-Token” Header I am sending in the request.

Just like backend has a way to convey which “origins” are allowed to see its responses (through the Access-Control-Allow-Origin header), the backend also has a way to convey which “Headers” are allowed to be sent to it.
In my case, nowhere did I program the backend to convey allowing “X-Access-Token” headers to the browser.
Fix: Set the Access-Control-Allow-Headers: * to all the responses sent from the backend to tell your browser that attaching any header to the request is fine, including the “X-Access-Token” header.

Note the “Access-Control-Allow-Headers” change I made to allow any incoming headers including X-Access-Token

Tests, again?

I pushed this code and told my friend working on frontend that I fixed it.

That’s it!…. You sure about that?

I get a message from my friend again, “It doesn’t work!”

I was shocked to see what was happening this time. And unfortunately,

So, I asked my friend to show me the exact request he was sending again. Initially, it looked the same but after looking thoroughly into it, I saw that he wasn’t making the “exact” request the backend was programmed with.

The request he was making was GET /v1/loan/proposals .
But the backend was programmed with GET /v1/loan/proposals/

Noticed the difference?
The request he was making did not end with a slash (trailing slash) but the backend is programmed to respond to the one with a trailing slash

But if that’s the case, why didn’t the backend respond with a 404 NotFound? I tried this locally on my API testing client by executing the exact request which my friend was making (without the trailing slash)

Interesting, turns out that the backend can notice the absence of the trailing slash and accordingly, redirect the request to the right endpoint.

GET /v1/loan/proposals becomes GET /v1/loan/proposals/ automatically.

Back to BBT? Browser-Based Testing 🤓?

Dang! That’s the first error we noticed.
This meant that the browser is not receiving the response with the right CORS headers.

But that shouldn’t be possible, I applied that middleware (attaching all the rightful headers) to my backend’s rootmost routing engine and it should set those response headers for the all responses it sends.

Also, if you look at the second-last screenshot (where I tested through the API testing client), you can see that the redirected request’s response still ended up getting attached with the headers Access-Control-Allow-Origin: * and Access-Control-Allow-Headers: * .

So why is the browser not receiving it? 😡

Time to learn something more!

I started some digging into how request re-direction works!

Source: https://developer.mozilla.org/en-US/docs/Web/HTTP/Redirections

This makes sense! So it’s not like the backend automatically internally redirects the request.

Actually, the backend responds to the browser with,

“301: Hey, /v1/loan/proposalsdoesn’t exist but /v1/loan/proposals/exists, please redirect your request to the right endpoint.”

And then, the browser/client looks at this response and its status code (301 or 307) and realises that this is a “redirection” situation and it should automatically redirect the request to the newer endpoint.

This is exactly why the redirection perfectly works on local API testing client.

So what’s the issue?

There’s only one possibility: when the backend responds with a 301/307 to the browser suggesting to redirect the request, the response doesn’t contain the CORS headers I programmed.

“But as I mentioned previously: I applied the header-attaching middleware for ALL requests/responses. So, even the redirection response should have gotten those headers!!!!”

I came to know that there’s a deeper issue happening behind the scenes!

Time to pull out the big guns

Enter the debugger!

Let’s place a breakpoint (note that red dot at line 41) at the line of code where the middleware for attaching headers to all responses starts executing.

Time to go to the browser and fire the same request

Dang! The browser instantly got a response and the code didn’t even get stuck on the breakpoint.
Either something’s wrong with the browser or something’s wrong with my debugger.

So, I proceeded to fire the same GET /v1/loan/proposals request via my local API testing client (with no CORS checks) and it got stuck at line 41.

This confirms one thing that the browser’s request didn’t even reach this middleware. Meaning that the response it got never got the opportunity to get attached with the right response headers.

Now, at this breakpoint, we have access to the ctx variable, which contained all the information about both the request and response until that moment.

So my expectation was to look inside the ctx variable’s “request” section and see that the request URL is /v1/loan/proposals (without the trailing slash).

But, I see something weird!

Yo what the heck! The request directly points to /v1/loan/proposals/ (already redirected request) instead of the originally requested v1/loan/proposols(without the trailing slash)

This means that the middleware I wrote to save us from the CORS issue is not even getting the opportunity to get executed when returning the first redirection response (with 301/307).

This explains the issue why the browser is facing the CORS issue with redirection without the breakpoint even executing.
So, from the eyes of the browser, when it makes a call to GET /v1/loan/proposals , the backend responds with 301 Moved Permanently to /v1/loan/proposals/ without applying the middleware I wrote and hence, not setting any response headers.
Remember, the browser applies CORS check to every response it receives, which includes the 301 response my backend sent to it. That’s where it noticed that there is no Access-Control-Allow-Origin: * header on it.
Hence, it straightaway gave this error, instead of redirecting to /v1/loan/proposals.
But why does this work with my local API testing client (like POSTman)?
That is because CORS is a browser level problem. The local API testing client upon making the request gets a 301 Moved Permanently to /v1/loan/proposals/ . But unlike the browser, it doesn’t perform any CORS check and directly fires a new request pointing to /v1/loan/proposals/ and this gets captured by the middleware I wrote, hence, pausing the backend’s execution at the breakpoint I attached.
This is why debugger shows that the breakpoint directly sees GET /v1/loan/proposals/ for the first time itself.

But the question still remains

“I programmed the server to apply that header to all the responses”
“Why isn’t it applying this to the 301 redirection responses too?”

Seems like something “else” is wrong!

Time to go deeper 👀…… in the server-side framework!

After diggin’ through the server-side framework’s source code, I found out that the every request is gated by a method called ServeHTTP(..., req Request)

Source: https://github.com/gin-gonic/gin/blob/v1.9.1/gin.go#L570

As this is the bottommost level where every request enters, I just applied a breakpoint here to see each request passing through it. This way I should be able capture and pause all the requests, including the original request requiring redirection GET /v1/loan/proposals. And accordingly, I should be able to step-by-step see how that request is processed by the server-side framework and further redirected.

Let’s fire that request

Whatttt!!!!! The code didn’t stop at that breakpoint!

The code should have paused at that breakpoint because every request must go through it!
It was as if the request didn’t reach the backend in the first place and the browser just assumed this endpoint returns CORS error.

Another interesting observation was that the browser was returning this error instantly, instead of having a little network-overhead related slowness.

This made me have a random hunch that maybe the browser is caching this CORS error. Because in the past as well, I had made this exact same request so the browser just cached the old response of CORS error.

Cache, you devil!

I did some digging and BINGO!!! That is the case.

I had to disable the network cache

And guess what, after firing the request again, the backend received the request 🎉

Yay! At the breakpoint, I finally captured the pre-flight OPTIONS request pointing to the wrong /v1/loan/proposals endpoint (without the trailing slash).

So, I resumed the debugger line-by-line and found that the preflight requests don’t get redirected because it’s an OPTIONS request.
The redirection seems to happen only for original (after preflight) requests.

This means that my backend treats the preflight request through the path of a 404 NotFound because my backend wasn’t programmed to know about any GET /v1/loan/proposals endpoint (without trailing slash).

But just before responding a 404, the preflight request goes through the middleware I wrote because remember, my middleware executes for all requests it receives.

Remember this screenshot? ;) Note the OPTIONS path carefully, it is the original one and hasn’t been redirected because it’s a preflight request.

Note, how my middleware, looks at the incoming preflight request (originally a 404), realises that it’s an OPTIONS request, and therefore, responds with a 204 while attaching the rightful response headers.

This preflight request’s response reaches the frontend browser with the right response headers.
The browser happily accepts it and proceeds to make the actual GET /v1/loan/proposals (still without the trailing slash because no redirection had happened yet).

Let’s resume the debugger to see that

See, the bottommost breakpoint of the server-side framework received another request and as expected, this time at GET /v1/loan/proposals

Now let’s resume line-by-line to see where does the redirection happen.

https://github.com/gin-gonic/gin/blob/8790d08909fc4d193c6c787c9c72f3089168f411/gin.go#L639

Woah, I think I am very close. The server seems to realise that the request GET /v1/loan/proposals doesn’t end with a trailing slash, hence, it seems to try redirecting it to /v1/loan/proposals/ via the function redirectTrailingSlash(c) where c consists of all the information about the request and response.

Seems like more or less, this function redirectTrailingSlash(c) cleans the path of the request and attaches a trailing slash at its end.

Then, it calls redirectRequest(c) (c has the modified path now with a trailing slash).

Let’s proceed further to dig into the redirectRequest(c) function

Woaaahhh!!! Found the issue

Look how this method is calling redirection natively via http.Redirect which is through the standard network library of Golang.

No where it is applying the middlewares I programmed. It is just sending the bare naked request to http.Redirect directly with no CORS headers.

Now, it all makes sense.
That’s why the redirection response didn’t go through my middleware and have the headers which I programmed initially.

So, it’s the framework’s issue! Now what?

Open-Source for the win.

I raise an issue and a pull request. The pull request essentially re-programs the framework to allow registering middlewares to Redirected requests as well.

I couldn’t wait for them to merge it, so I made my backend use my fork (with the fix) as a dependency.

go mod edit -replace="github.com/gin-gonic/gin=github.com/yashvardhan-kukreja/gin@issue-3857-onredirect-middleware" && \
go mod tidy

Finally, I re-programmed my backend to use the new method with the fix (raised as a pull request as well) to register my middleware on request redirection as well.

You won’t find the OnRedirect method in gin. You can only find it in my fork here — https://github.com/yashvardhan-kukreja/gin/blob/issue-3857/onredirect-middleware/gin.go#L302-L306

BBT Again!

Making a request with the trailing slash (the one which the backend always worked with)

Making a request without the trailing slash (the one which my friend was making and it required redirection)

Woohoo! Works like a charm!

Let’s summarize!

When an endpoint like /v1/loan/proposals doesn’t exist in the backend but /v1/loan/proposals/ does, the backend responds with 301 response instead of 404 to tell the client (browser) to redirect the original request instead.
The browser expects Access-Control-Allow-Origin: * and Access-Control-Allow-Headers: * to be written on all the responses including the 301 response.
In my case, the backend wrote those headers to all responses except the 301 response which it sent to the browser telling it to redirect the original requests.
Fix was to fix the server-side framework to write those headers to redirection response 301 as well.

Woah! This was a big read

This marks the end of this 2-part article series.

Thanks for reaching until this point.

I get it that this was a long read but I hope you saw how we navigated all the way across the browser, various docs around how requests work internally, the server-side framework’s source code and learned to use debugger to our advantage to fight and defeat the infamous CORS issue.

I strongly believe that any time in the future if/when you’d face any CORS issue, you’d be easily able to troubleshoot it.

Until next time,

Adios

References

Any Questions?

Feel absolutely free to drop in any questions or comments or even reach out to me on any of my social media handles:

Twitter — https://twitter.com/yashkukreja98

Github — https://github.com/yashvardhan-kukreja

LinkedIn — https://www.linkedin.com/in/yashvardhan-kukreja (expect slow replies here as I mostly stay inactive on it :P)