In ASP.NET, the customErrors configuration element is used to handle error situations. However, the behaviour of the custom errors is somewhat counterintuitive, as you might end up with your error pages indexed by search engines.
How customErrors work
First a quick example of how a customError section might look like in a web.config file (this belongs under system.web):
<customErrors mode="RemoteOnly" defaultRedirect="/StandardError.aspx"> <error statusCode="404" redirect="/NotFound.aspx"/> </customErrors>
CustomErrors can be set to: Off | RemoteOnly | On. Off means that you'll get the default (and detailed) ASP.NET error pages when something bad happens. Remoteonly will give the default error pages when the application is accessed from localhost, but will serve your custom pages for requests not originating on the local machine. On will always serve the custom error pages.
Historically, errors would trigger a redirect to en error page — this is still the default behaviour. The path to the page that triggered the error is included as a parameter:
Starting with the .Net framework 3.51, the customErrors element includes the optional redirectMode attribute. When setting this to "ResponseRewrite", the user is no longer redirected to the error page. Instead, the error page is served "in place" on the page where the error occured. This can be advantageous, as the user can simply refresh the page to try again instead of being sent away from the webpage where she's trying to accomplish something.
Though it's important to present a professional looking error page there is also important behaviour invisible to the average end-user: HTTP status codes affecting how search engines index your site.
HTTP status codes
HTTP status codes are fundamental to the functioning of the web. In short, they're numeric values describing the outcome of a request. The status codes are included in the first line of the response by a webserver. Here's the most common ones:
- 200 OK
- 301 Moved permanently (this is a redirect)
- 302 Moved temporarily (this is a redirect)
- 404 Not found
- 500 Internal server error
When running with customErrors Off, a request for a non-existant aspx will yield a default ASP.NET "Server error, file not found" error page, correctly returned with a 404 HTTP status code. This behaviour is important, as the 404 status code indicates to search engines that the resource did not exist.
However, it all changes when the customErrors are set to RemoteOnly or On! You'll see your custom error page served, but the status code in the response will be 200. This indicates that everything went well! Search engines will consequently index your error page at will — and they will keep returning to the address to check for updates. Why the behaviour changes to return a 200 instead of a 404 is beyond me. In my opinion, it shouldn't. We'll revisit our example to show just how counterintuitive this is, we've now included the ResponseRewrite functionality:
<customErrors mode="RemoteOnly" defaultRedirect="/StandardError.aspx" redirectMode="ResponseRewrite"> <error statusCode="404" redirect="/NotFound.aspx"/> </customErrors>
Note that we are specific about the 404 errors — we even refer it by it's numerical code — but the error page will still be returned with a 200 OK.
Fixing the problem
For the NotFound.aspx error page in our example, the statuscode can be set programatically in e.g. the Page_Load() event :
Response.StatusCode = 404;
This will override the status code in the response, and make your "file not found" custom error page behave correctly!
As a sidenote, there are several tricks to keep your regular pages out of the search engine indexes. Google has published several articles on how to keep stuff out of their index, check them out!