Tag: nginx
2011
08.24

After publishing my previous blog post on PHP, nginx configuration, and potential arbitrary code execution, I came across a separate null-byte injection vulnerability in older versions of nginx (0.5.*, 0.6.*, 0.7 <= 0.7.65, 0.8 <= 0.8.37). By taking advantage of this vulnerability, an attacker can cause a server that uses PHP-FastCGI to execute any publicly accessible file on the server as PHP.

In vulnerable versions of nginx, null bytes are allowed in URIs by default (their presence is indicated via a variable named zero_in_uri defined in ngx_http_request.h). Individual modules have the ability to opt-out of handling URIs with null bytes. However, not all of them do; in particular, the FastCGI module does not.

The attack itself is simple: a malicious user who makes a request to http://example.com/file.ext%00.php causes file.ext to be parsed as PHP. If an attacker can control the contents of a file served up by nginx (ie: using an avatar upload form) the result is arbitrary code execution. This vulnerability can not be mitigated by nginx configuration settings like try_files or PHP configuration settings like cgi.fix_pathinfo: the only defense is to upgrade to a newer version of nginx or to explicitly block potentially malicious requests to directories containing user-controlled content.

1
2
3
4
5
6
# This location block will prevent an attacker from exploiting
# this vulnerability using files in the 'uploads' or 'other_uploads' directory
location ~ ^/(uploads|other_uploads)/.*.php$
{
    deny all;
}

Although the affected versions of nginx are relatively old (0.7.66 was released June 7th, 2010, 0.8.38 was released May 24th 2010), no mention of the change appears in the release notes. As a result, administrators may be running vulnerable servers without realizing their risk. I discovered a couple places where vulnerable packages were being distributed:

  1. Ubuntu Lucid Lynx (Ubuntu’s current LTS offering) and Hardy Heron (via both the hardy and hardy-backports repositories) provided vulnerable versions of nginx via apt-get. The lucid and hardy packages have been updated: hardy-backports is awaiting approval. [1] [2]
  2. Fedora provides a vulnerable version in its EPEL-4 repository. At this time, an updated package has not been released.

I sent several emails to igor@sysoev.ru regarding the vulnerability. I sent the first on June 24th and I sent followups on July 4th, July 20th, and August 2nd. I received the following reply to my August 2nd email:

Thank you for report.

I do not consider this as nginx security issue since every application
should validate its input data, so nginx passed the data to application.
Also this is PHP installation issue where scripts and user uploaded
data are not separated. This issue was discussed several times on mailing
 list.

At some point I’ve decided that zero byte in URI should not appear
in any encoding, operating system, etc., and just makes more problems
than helps. So I have remove zero byte test.

For anyone who’s curious, the changes can be found at r3528 from svn://svn.nginx.org. At that time, it appears trunk corresponded to nginx 0.8: r3599 merged r3528 into the nginx 0.7 branch. The corresponding commit message is “remove r->zero_in_uri.” I’ve reproduced the output of svn diff below:

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
#!/usr/bin/diff
Index: src/http/ngx_http_request.h
===================================================================
--- src/http/ngx_http_request.h (revision 3527)
+++ src/http/ngx_http_request.h (revision 3528)
@@ -56,7 +56,7 @@
 #define NGX_HTTP_PARSE_INVALID_HEADER      13


-#define NGX_HTTP_ZERO_IN_URI               1
+/* unused                                  1 */
 #define NGX_HTTP_SUBREQUEST_IN_MEMORY      2
 #define NGX_HTTP_SUBREQUEST_WAITED         4
 #define NGX_HTTP_LOG_UNSAFE                8
@@ -435,9 +435,6 @@
     /* URI with "+" */
     unsigned                          plus_in_uri:1;

-    /* URI with "\0" or "%00" */
-    unsigned                          zero_in_uri:1;
-
     unsigned                          invalid_header:1;

     unsigned                          valid_location:1;
Index: src/http/ngx_http_core_module.c
===================================================================
--- src/http/ngx_http_core_module.c (revision 3527)
+++ src/http/ngx_http_core_module.c (revision 3528)
@@ -1341,7 +1341,7 @@

     /* no content handler was found */

-    if (r->uri.data[r->uri.len - 1] == '/' && !r->zero_in_uri) {
+    if (r->uri.data[r->uri.len - 1] == '/') {

         if (ngx_http_map_uri_to_path(r, &path, &root, 0) != NULL) {
             ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
@@ -2104,7 +2104,6 @@
     ngx_log_debug2(NGX_LOG_DEBUG_HTTP, c->log, 0,
                    "http subrequest \"%V?%V\"", uri, &sr->args);

-    sr->zero_in_uri = (flags & NGX_HTTP_ZERO_IN_URI) != 0;
     sr->subrequest_in_memory = (flags & NGX_HTTP_SUBREQUEST_IN_MEMORY) != 0;
     sr->waited = (flags & NGX_HTTP_SUBREQUEST_WAITED) != 0;

Index: src/http/ngx_http_special_response.c
===================================================================
--- src/http/ngx_http_special_response.c    (revision 3527)
+++ src/http/ngx_http_special_response.c    (revision 3528)
@@ -517,8 +517,6 @@

     r->err_status = overwrite;

-    r->zero_in_uri = 0;
-
     if (ngx_http_complex_value(r, &err_page->value, &uri) != NGX_OK) {
         return NGX_ERROR;
     }
Index: src/http/ngx_http_upstream.c
===================================================================
--- src/http/ngx_http_upstream.c    (revision 3527)
+++ src/http/ngx_http_upstream.c    (revision 3528)
@@ -1815,10 +1815,6 @@
             return NGX_DONE;
         }

-        if (flags & NGX_HTTP_ZERO_IN_URI) {
-            r->zero_in_uri = 1;
-        }
-
         if (r->method != NGX_HTTP_HEAD) {
             r->method = NGX_HTTP_GET;
         }
Index: src/http/ngx_http_parse.c
===================================================================
--- src/http/ngx_http_parse.c   (revision 3527)
+++ src/http/ngx_http_parse.c   (revision 3528)
@@ -438,8 +438,7 @@
                 r->plus_in_uri = 1;
                 break;
             case '\0':
-                r->zero_in_uri = 1;
-                break;
+                return NGX_HTTP_PARSE_INVALID_REQUEST;
             default:
                 state = sw_check_uri;
                 break;
@@ -496,8 +495,7 @@
                 r->plus_in_uri = 1;
                 break;
             case '\0':
-                r->zero_in_uri = 1;
-                break;
+                return NGX_HTTP_PARSE_INVALID_REQUEST;
             }
             break;

@@ -526,8 +524,7 @@
                 r->complex_uri = 1;
                 break;
             case '\0':
-                r->zero_in_uri = 1;
-                break;
+                return NGX_HTTP_PARSE_INVALID_REQUEST;
             }
             break;

@@ -1202,7 +1199,7 @@
                     ch = *p++;

                 } else if (ch == '\0') {
-                    r->zero_in_uri = 1;
+                    return NGX_HTTP_PARSE_INVALID_REQUEST;
                 }

                 state = quoted_state;
@@ -1304,8 +1301,7 @@
         }

         if (ch == '\0') {
-            *flags |= NGX_HTTP_ZERO_IN_URI;
-            continue;
+            goto unsafe;
         }

         if (ngx_path_separator(ch) && len > 2) {
@@ -1449,34 +1445,19 @@
 void
 ngx_http_split_args(ngx_http_request_t *r, ngx_str_t *uri, ngx_str_t *args)
 {
-    u_char  ch, *p, *last;
+    u_char  *p, *last;

-    p = uri->data;
+    last = uri->data + uri->len;

-    last = p + uri->len;
+    p = ngx_strlchr(uri->data, last, '?');

-    args->len = 0;
+    if (p) {
+        uri->len = p - uri->data;
+        p++;
+        args->len = last - p;
+        args->data = p;

-    while (p < last) {
-
-        ch = *p++;
-
-        if (ch == '?') {
-            args->len = last - p;
-            args->data = p;
-
-            uri->len = p - 1 - uri->data;
-
-            if (ngx_strlchr(p, last, '\0') != NULL) {
-                r->zero_in_uri = 1;
-            }
-
-            return;
-        }
-
-        if (ch == '\0') {
-            r->zero_in_uri = 1;
-            continue;
-        }
+    } else {
+        args->len = 0;
     }
 }
Index: src/http/modules/ngx_http_gzip_static_module.c
===================================================================
--- src/http/modules/ngx_http_gzip_static_module.c  (revision 3527)
+++ src/http/modules/ngx_http_gzip_static_module.c  (revision 3528)
@@ -89,10 +89,6 @@
         return NGX_DECLINED;
     }

-    if (r->zero_in_uri) {
-        return NGX_DECLINED;
-    }
-
     gzcf = ngx_http_get_module_loc_conf(r, ngx_http_gzip_static_module);

     if (!gzcf->enable) {
Index: src/http/modules/ngx_http_index_module.c
===================================================================
--- src/http/modules/ngx_http_index_module.c    (revision 3527)
+++ src/http/modules/ngx_http_index_module.c    (revision 3528)
@@ -116,10 +116,6 @@
         return NGX_DECLINED;
     }

-    if (r->zero_in_uri) {
-        return NGX_DECLINED;
-    }
-
     ilcf = ngx_http_get_module_loc_conf(r, ngx_http_index_module);
     clcf = ngx_http_get_module_loc_conf(r, ngx_http_core_module);

Index: src/http/modules/ngx_http_random_index_module.c
===================================================================
--- src/http/modules/ngx_http_random_index_module.c (revision 3527)
+++ src/http/modules/ngx_http_random_index_module.c (revision 3528)
@@ -86,10 +86,6 @@
         return NGX_DECLINED;
     }

-    if (r->zero_in_uri) {
-        return NGX_DECLINED;
-    }
-
     if (!(r->method & (NGX_HTTP_GET|NGX_HTTP_HEAD|NGX_HTTP_POST))) {
         return NGX_DECLINED;
     }
Index: src/http/modules/ngx_http_dav_module.c
===================================================================
--- src/http/modules/ngx_http_dav_module.c  (revision 3527)
+++ src/http/modules/ngx_http_dav_module.c  (revision 3528)
@@ -146,10 +146,6 @@
     ngx_int_t                 rc;
     ngx_http_dav_loc_conf_t  *dlcf;

-    if (r->zero_in_uri) {
-        return NGX_DECLINED;
-    }
-
     dlcf = ngx_http_get_module_loc_conf(r, ngx_http_dav_module);

     if (!(r->method & dlcf->methods)) {
Index: src/http/modules/ngx_http_flv_module.c
===================================================================
--- src/http/modules/ngx_http_flv_module.c  (revision 3527)
+++ src/http/modules/ngx_http_flv_module.c  (revision 3528)
@@ -80,10 +80,6 @@
         return NGX_DECLINED;
     }

-    if (r->zero_in_uri) {
-        return NGX_DECLINED;
-    }
-
     rc = ngx_http_discard_request_body(r);

     if (rc != NGX_OK) {
Index: src/http/modules/ngx_http_static_module.c
===================================================================
--- src/http/modules/ngx_http_static_module.c   (revision 3527)
+++ src/http/modules/ngx_http_static_module.c   (revision 3528)
@@ -66,10 +66,6 @@
         return NGX_DECLINED;
     }

-    if (r->zero_in_uri) {
-        return NGX_DECLINED;
-    }
-
     log = r->connection->log;

     /*
Index: src/http/modules/ngx_http_autoindex_module.c
===================================================================
--- src/http/modules/ngx_http_autoindex_module.c    (revision 3527)
+++ src/http/modules/ngx_http_autoindex_module.c    (revision 3528)
@@ -160,10 +160,6 @@
         return NGX_DECLINED;
     }

-    if (r->zero_in_uri) {
-        return NGX_DECLINED;
-    }
-
     if (!(r->method & (NGX_HTTP_GET|NGX_HTTP_HEAD))) {
         return NGX_DECLINED;
     }
Index: src/http/modules/perl/ngx_http_perl_module.c
===================================================================
--- src/http/modules/perl/ngx_http_perl_module.c    (revision 3527)
+++ src/http/modules/perl/ngx_http_perl_module.c    (revision 3528)
@@ -168,10 +168,6 @@
 static ngx_int_t
 ngx_http_perl_handler(ngx_http_request_t *r)
 {
-    if (r->zero_in_uri) {
-        return NGX_HTTP_NOT_FOUND;
-    }
-
     r->main->count++;

     ngx_http_perl_handle_request(r);
2011
04.07

Summary

Several days ago, I had to deal with a compromised web application: an attacker had somehow managed to upload PHP backdoor scripts onto the application’s server. Thanks to some log file sleuthing and Google searches, I was quickly able to identify what had allowed the attack: a misconfigured nginx server can allow non-PHP files to be executed as PHP. As I researched the vulnerability a bit more, however, I realized that many of the nginx / PHP setup tutorials found on the Internet suggest that people use vulnerable configurations.

The misconfiguration

As I mentioned, the attack was made possible by a very simple misconfiguration between nginx and php-fastcgi. Consider the configuration block below, taken from a tutorial at http://library.linode.com/web-servers/nginx/php-fastcgi/fedora-14:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
server {
    listen 80;
    server_name www.bambookites.com bambookites.com;
    access_log /srv/www/www.bambookites.com/logs/access.log;
    error_log /srv/www/www.bambookites.com/logs/error.log;
    root /srv/www/www.bambookites.com/public_html;

    location / {
        index  index.html index.htm index.php;
    }

    location ~ \.php$ {
        include /etc/nginx/fastcgi_params;
        fastcgi_pass  127.0.0.1:9000;
        fastcgi_index index.php;
        fastcgi_param  SCRIPT_FILENAME /srv/www/www.bambookites.com/public_html$fastcgi_script_name;
    }
}

It may not be immediately clear, but this configuration block allows for arbitrary code execution under certain circumstances (and I don’t just mean if an attacker can upload a file ending in .php: that kind of vulnerability is independent of the web server used).

Consider a situation where remote users can upload their own pictures to the site. Lets say that an attacker uploads an image to http://www.bambookites.com/uploads/random.gif. What happens, given the server block above, if the attacker then browses to http://www.bambookites.com/uploads/random.gif/somefilename.php?

  1. nginx will look at the URL, see that it ends in .php, and pass the path along to the PHP fastcgi handler.
  2. PHP will look at the path, find the .gif file in the filesystem, and store /somefilename.php in $_SERVER['PATH_INFO'], executing the contents of the GIF as PHP.

Since GIFs and other image types can contain arbitrary content within them, it’s possible to craft a malicious image that contains valid PHP. That is how the attacker was able to compromise the server: he or she uploaded a malicious image containing PHP code to the site, then browsed to the file in a way that caused it to be parsed as PHP.

This issue was first discovered almost a year ago. The original report can be found in Chinese at http://www.80sec.com/nginx-securit.html. There is also a discussion about it on the nginx forums.

This issue can be mitigated in a number of ways, but there are downsides associated with each of the possibilities:

  1. Set cgi.fix_pathinfo to false in php.ini (it’s set to true by default). This change appears to break any software that relies on PATH_INFO being set properly (eg: Wordpress).
  2. Add try_files $uri =404; to the location block in your nginx config. This only works when nginx and the php-fcgi workers are on the same physical server.
  3. Add a new location block that tries to detect malicious URLs. Unfortunately, detecting based on the URL alone is impossible: files don’t necessarily need to have extensions (eg: README, INSTALL, etc).
  4. Explicitly exclude upload directories using an if statement in your location block. The disadvantage here is the use of a blacklist: you have to keep updating your nginx configuration every time you install a new application that allows uploads.
  5. Don’t store uploads on the same server as your PHP. The content is static anyway: serve it up from a separate (sub)domain. Of course, this is easier said than done: not all web applications make this easy to do.

[Note: If anyone is aware of other possible solutions (or workarounds to improve the effectiveness of these solutions), please let me know and I’ll add them here!]

Tutorials

Now, the configuration file for the compromised server wasn’t written by hand. When the server was set up, the configuration was created based on suggestions found on the Internet. I assume that other people use tutorials and walkthroughs for setting up their servers as well. Unfortunately, most of the documentation for configuring nginx and php-fastcgi still encourages people to set up their servers in a vulnerable way.

  1. The default configuration file for nginx suggests the use of an insecure location block (source).
  2. The nginx wiki supplies potentially dangerous examples as well. To be fair, some pages do encourage users to explicitly prevent PHP execution in upload directories [Edit: and in the Pitfalls document, which everyone should read before configuring nginx]. However, other pages ignore the issue entirely.
  3. The Linode Library has an extensive collection of documents, including a number that talk about setting up nginx and PHP on various OSes. Unfortunately, all of those tutorials suggest using a vulnerable configuration for PHP. I’ve contacted the documentation team at Linode and I’m waiting to hear back from them. [Update: The Linode documentation team has updated the tutorials with more information and workarounds]
  4. Howto Forge has several tutorials (1, 2) which show up when searching Google for “nginx php setup.” These tutorials also suggest the use of a vulnerable configuration.
  5. People have written many tutorials on blogs and other sites (ie: 1, 2). A number of these tutorials encourage using the same vulnerable configuration.

In contrast, codex.wordpress.org provides an excellent configuration example that warns people about and mitigates the vulnerability. I’ve reproduced the relevant portion below:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
# Pass all .php files onto a php-fpm/php-fcgi server.
location ~ \.php$ {
   # Zero-day exploit defense.
   # http://forum.nginx.org/read.php?2,88845,page=3
   # Won't work properly (404 error) if the file is not stored on this server, which is entirely possible with php-fpm/php-fcgi.
   # Comment the 'try_files' line out if you set up php-fpm/php-fcgi on another machine.  And then cross your fingers that you won't get hacked.
   try_files $uri =404;

   fastcgi_split_path_info ^(.+\.php)(/.+)$;
   include fastcgi_params;
   fastcgi_index index.php;
   fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
#    fastcgi_intercept_errors on;
   fastcgi_pass php;
}

Conclusion

  1. If you run PHP on an nginx web server, check your configuration and update if necessary.
  2. If you’re doing a security audit on a PHP application running on an nginx web server, remember to test for this configuration.
  3. If you run across a tutorial that is out of date, please point the author to this post.
  4. If you know of a way to better secure nginx / php-fastcgi, let me know!

Edit: relix, a redditor, has pointed out that nginx lists this exact issue in the “Pitfalls” page on their wiki. I encourage everyone to read through that page!