This blog entry will attempt to describe the symptoms and likely causes of a collection of error messages often found to occur together in a given context.


Symptom:

[Browser Error Message, CFMX 6.0]:

Server Error
The server encountered an internal error and was unable to complete your request.
Too many concurrent requests, jcp.endpoint.main.max.threads exceeded.


[Browser Error Message, CFMX 6.1]:
Server Error
The server encountered an internal error and was unable to complete your request.
Application server is busy. Either there are too many concurrent requests or the server is still starting up.


Explanation:
Here, the error message changed between CFMX 6.0 and CFMX 6.1 to be somewhat more accurate regarding the cause. Essentially, there are three possible conditions in which you can get either of these errors.

  1. 1) ColdFusion Server is just starting up when the request was made. When ColdFusion is started, first the JRun server is started and then the ColdFusion Application Server (which is actually a deployed J2EE application in JRun) is started, and then your web application will become available when those two are complete. The above browser errors will be returned if your request comes in after JRun started but before ColdFusion is fully deployed.


  2. The Running Request pool (Simultaneous Requests) is saturated with long running requests, which also described as "the server is hung". This will cause new requests to be queued up. Then one of these two conditions has occurred:



    A) The queue has already reached its maximum size. This is a setting in jrun.xml JRun Proxy Service called "maxHandlerThreads". Your request could not be queued.

    B) Your request has been queued, but it has been waiting in the queue for a period exceeding the queue timeout. This is a setting in jrun.xml JRun Proxy Service called "threadWaitTimeout", and it's unit is seconds. jrun.xml



Additional Symptoms:[Apache error_log]
[Tue Oct 7 12:10:16 2003] [notice] jrApache[1745] PROXY_BUSY <- [11]
[Tue Oct 7 12:10:16 2003] [notice] jrApache[1745] returning error page for JRun too busy or out of memory
[Tue Oct 7 12:10:16 2003] [notice] jrApache[1731] PROXY_BUSY <- [3]
[Tue Oct 7 12:10:16 2003] [notice] jrApache[1731] returning error page for JRun too busy or out of memory
[Tue Oct 7 12:10:16 2003] [notice] jrApache[1746] PROXY_BUSY <- [14]
[Tue Oct 7 12:10:16 2003] [notice] jrApache[1746] returning error page for JRun too busy or out of memory
[Tue Oct 7 12:10:16 2003] [notice] jrApache[1732] PROXY_BUSY <- [3]
[Tue Oct 7 12:10:16 2003] [notice] jrApache[1732] returning error page for JRun too busy or out of memory
[Tue Oct 7 12:10:16 2003] [notice] jrApache[1748] PROXY_BUSY <- [8]
[Tue Oct 7 12:10:16 2003] [notice] jrApache[1748] returning error page for JRun too busy or out of memory
[Tue Oct 7 12:10:16 2003] [notice] jrApache[1749] PROXY_BUSY <- [9]
[Tue Oct 7 12:10:16 2003] [notice] jrApache[1749] returning error page for JRun too busy or out of memory
[Tue Oct 7 12:10:17 2003] [notice] jrApache[1750] PROXY_BUSY <- [6]
[Tue Oct 7 12:10:17 2003] [notice] jrApache[1750] returning error page for JRun too busy or out of memory
[Tue Oct 7 12:10:17 2003] [notice] jrApache[1752] PROXY_BUSY <- [12]
[Tue Oct 7 12:10:17 2003] [notice] jrApache[1752] returning error page for JRun too busy or out of memory
[Tue Oct 7 12:10:17 2003] [notice] jrApache[1753] PROXY_BUSY <- [5]
[Tue Oct 7 12:10:17 2003] [notice] jrApache[1753] returning error page for JRun too busy or out of memory


[Apache access_log]
192.168.0.2 - - [07/Oct/2003:12:10:16 -0400] "GET /serat/sleep.cfm?sec=120 HTTP/1.1" 503 236
192.168.0.2 - - [07/Oct/2003:12:10:16 -0400] "GET /serat/sleep.cfm?sec=120 HTTP/1.1" 503 231
192.168.0.2 - - [07/Oct/2003:12:10:16 -0400] "GET /serat/sleep.cfm?sec=120 HTTP/1.1" 503 236
192.168.0.2 - - [07/Oct/2003:12:10:16 -0400] "GET /serat/sleep.cfm?sec=120 HTTP/1.1" 503 231
192.168.0.2 - - [07/Oct/2003:12:10:16 -0400] "GET /serat/sleep.cfm?sec=120 HTTP/1.1" 503 236
192.168.0.2 - - [07/Oct/2003:12:10:16 -0400] "GET /serat/sleep.cfm?sec=120 HTTP/1.1" 503 236
192.168.0.2 - - [07/Oct/2003:12:10:17 -0400] "GET /serat/sleep.cfm?sec=120 HTTP/1.1" 503 236
192.168.0.2 - - [07/Oct/2003:12:10:17 -0400] "GET /serat/sleep.cfm?sec=120 HTTP/1.1" 503 236
192.168.0.2 - - [07/Oct/2003:12:10:17 -0400] "GET /serat/sleep.cfm?sec=120 HTTP/1.1" 503 236


Recommendations:

You should examine the performance of your application and review the settings configuration for the ColdFusion Administrator. You could start by logging slow pages, a setting in the CFAdmin, as well as generate some thread dumps.


See this blog entry for how to generate and debug "Full Thread Dumps" in CFMX, as well as pointers on where to look for tuning information.