JVM Memory Management and ColdFusion Log Analysis

The following is a document I wrote for knowledge sharing with some peers, but I feel that it might have some value to other ColdFusion Devs, Testers, and Admins out there. The purpose was to illustrate how I went about analyzing CF's performance during a prior troubleshooting session. I'm re-purposing the content here after scrubbing some private information. Hopefully it still makes sense, although slightly out of context.




These are some technical notes on what to look for when analyzing ColdFusion server performance history. It includes concepts and techniques to assess what performance related problems might exist, emphasizing memory usage issues first. This is a somewhat simplified explanation about how the JVM manages memory and its relation to CF applications, and about how I went about analyzing them. There are many similar resources on the web, but I found many of them are quite technical, so this article is written with more of a layman's approach to make it more digistible to those not as familiar with troubleshooting ColdFusion or Java apps.



The ColdFusion Application Server runs inside (is "contained" in) a higher level JRun J2EE server. The J2EE server (and therefore the CF server) run on top a JVM (Java Virtual Machine). To analyze the ColdFusion and JVM performance, I take a forensic approach. I start by collecting the ColdFusion and JRun server logs. I also colelct the JVM Garbage Collection (GC) log that has been manually enabled to log information regarding how the JVM is cleaning up the memory that is has used. The JVM is configured with an algorithm that tells it what approach to take when cleaning up and freeing memory. The application's Java objects (like queries, session variables, local variables, cfc instances, etc) are held in the JVM's memory. Objects are said to hold "references" in memory, meaning that something in the application is potentially using that object. When the application no longer has a need for an object, its memory is dereferenced. That dereferenced memory can be released by the JVM and then reused by other objects that require it.



[More]

Adobe LiveCycle DataServices for ColdFusion at CFObjective

Allaire's CEO, David OrfaoAfter a decade of working intensely with the ColdFusion server, I'm finally getting the courage to start presenting about it on the conference circuit. As a blogger, tweeter, and contributor to mailing lists I'm very confident helping others solve ColdFusion related problems because I can do that from the quiet comfort of my own desk. However, one of my greatest fears has always been public speaking. I'm the kind of person that feels like I need to know the subject matter cold, so that I can speak from the hip without relying on looking at the slides.




Blackstone Test CDsOver the years, I had some opportunities to present to small groups, and I recall each time feeling the adrenalin surge and my heart pounding. That started with presenting ColdFusion for Unix and Linux as an internal training class at Macromedia. Later, while taking classes at the Harvard Extension School, I was honored to be asked to present to CSCI-253 Developing Web-Based Database Applications. Even more so, I presented twice there in one year. The first time on Building ColdFusion Web Applications with CFEclipse and Dreamweaver, and later on ColdFusion Server Administration




MAX in ActionI've been attending ColdFusion conferences since the days of Allaire DevCon, but had never presented at any of them including MAX. My long time friend in the local ColdFusion Community, Brian Rinaldi, continued to encourage me to present at the local Boston CFUG as a starting point, as well as the new conference that he was organizing, RIA Unleashed, held in Bentley College this past November. The members of the CFUG were kind enough to let me present a draft of a presentation that I was to later give at RIA Unleashed. My presentation topic was Adobe LiveCycle DataServices Data Management for Mere Mortals


ColdFusion 1.5 on Floppy DisksFortunately at RIA Unleashed I was among the very first sessions after the keynote, so there was no time to build up butterflies that morning. If beforehand you would have told me that among the audience front row would be Ben Nadel, Simon Free, and Ray Camden with Tom Jordahl tucked way in the back then I surely would have freaked out. But they were both kind enough to chat with me before hand and even lend some technical assistance getting setup with the A/V, so that really put me at ease. With a firm limit of 50 minutes, I pushed all the way through what should have been a 90 minute talk, all the while trying to remember to speak clearly and loudly. The talk went off pretty much without a hitch as I found myself completely focused on the technical content and not at all worrying about the large room filled with people in front of me. I was delighted at the end when Tom complemented me on talk, which to me was the ultimate satisfaction.




First Unix machine to run ColdFusionI chose LCDS for ColdFusion as a topic because while I was a QA Engineer on the ColdFusion team at Adobe, I was paired with Tom, a Computer Scientist at Adobe who architected the integration between the products. Heck, Tom architected much of ColdFusion itself, and was in fact the original engineer to have ported ColdFusion to run on Unix and Linux back in the day. Tom is a font of information, and I cut my teeth on the feature under his guidance, which was then known as Flex Data Services and later renamed under the LiveCycle brand. I spent many days last summer and fall revisiting all the LCDS documentation again to ensure the quality of my presentation and to mentally prepare me for the upcoming conference.




ColdFusion Team, BangaloreWIth my first conference under my belt, I decided to throw my hat into the ring for the ultimate ColdFusion experience, CFObjective, which is promoted as The Only Enterprise ColdFusion Conference. I'm excited to announce that I have been selected to be a speaker at the conference, which runs from April 22-24th in Minneapolis, Minnesota. The conference is divided into three tracks for technologies related to ColdFusion. I'll be speaking the last day in the Flex track, once again on the topic of LiveCycle DataServices for ColdFusion Developers. Specifically I'll be talking about the prime feature of LCDS, the Data Management capabilities. With any luck I'll be updating my presentation to consider the benefits of working with the latest versions of Adobe software. Here's the brief description and the PDF:




Discussions of Adobe's LiveCycle Data Services are often entered with the same trepidation as those of Organic Chemistry or Quantum Mechanics, but with ColdFusion, building Web applications that manage complex data sets doesn't have to be that scary. Data Management is a pillar of LCDS that offers scalable, real-time data synchronization across very large numbers of connected clients with the benefits of conflict resolution and data pagination.† Come learn how to quickly get up to speed with Data Management by letting ColdFusion do the hard work for you.


If you're seriously interested in ColdFusion, then CFObjective is the conference for you. I hope to see you there.






ColdFusion Screams

Recent Tweets for Fri Oct 2, 2009 Part II

Follow me on Twitter!





Tue Sep 15 9:04 PM
@iotashan @rukumar Shan meet Rupesh. Rupesh meet Shan. You guys should talk CF9 ORM. ;-) Rupesh, Shan works with me & has an ORM issue
Tue Sep 15 8:51 PM
No CF Admin DSN setting for isolation level, but u can add SET TRANSACTION ISOLATION LEVEL <level> as u're validation query as workaround
Tue Sep 15 5:36 PM
Tue Sep 15 4:13 PM
@berniedolan Yup, and I was on a downhill at 35mph, slowed to 15 then skidded to within inches as he made a blinkerless right turn




[More]

Recent Tweets for Fri Oct 2, 2009

Follow me on Twitter!



Fri Oct 02 7:26 PM
Ghosts of ColdFusion Past http://yfrog.com/3omigoj
Fri Oct 02 4:40 PM
@john_mason_ Thanks. Indeed, the server was under high load.
Fri Oct 02 4:21 PM
Ditto that! RT @awest: Working at home really blows. Not. http://bit.ly/UflXy
Fri Oct 02 3:44 PM
@charliegriefer Enjoy! Twitter is gonna have a melt down. #adobeMAX
Fri Oct 02 3:12 PM
Have you ever launched the ColdFusion Server Monitor and seen the buttons for Monitoring, Profiling, and Memory just not show up at the top?




[More]

Recent Tweets for Fri August 28, 2009

Follow me on Twitter!





Fri Aug 28 8:52 PM
Why is it that ColdFusion must be started by a privileged user on every operating system but Mac? Hmph!
Fri Aug 28 8:43 PM
@shutupdanielle Could be worse, you could be in Bolton. 2 pizza shops, that's it. Nary a chicken wing in sight (really).
Fri Aug 28 8:36 PM
@mobarger Big city turn me loose, set me free, somewhere in the middle of Montana
Fri Aug 28 8:08 PM
@styggiti Outside temp last night was 52F here, but not sure which is better lower temps or lower humidity.
Fri Aug 28 5:43 PM
ColdFusion 9 (& ColdFusion 8.01) bug for CF Multiserver's bin/cf-init.sh file. Please vote: http://bit.ly/MsgCT
Fri Aug 28 4:56 PM
Yep, verified the 3 issues with the Solaris part of the cf-init.sh boot script installer for ColdFusion 9. Oh, bugtracker.....


[More]

I'm Twittering This! or Dude, Where's My Blog?

As you may have noticed, TalkingTree.com blog is rather barren these days. I find that while I work, while I relax at home (that is when I'm not working at home), or even when I'm mobile I'm nearly always using Twitter, and less so Facebook.

Unless you've been living under a rock you've heard of Twitter by now. If you're still not sure, Twitter is a bit like group chat where you post a short summary of what you're doing or what's on your mind, and other people that are interested can reply. While I'm not going to explain Twitter fully here -- you can find that on Twitter.com -- it can be very useful for sharing up to date information about a particular interest with lots of other people that share that interest. Most of my Tweeps fall into one of several interest groups... There's those interested in Photography, those interested in Web Technology, and those that happen to otherwise be friends or family. There are several applications you can use to watch Twitter updates. My favorite for use on a computer is TweetDeck, and on my iPhone I use Twitterific. One reason to love using an iPhone with Twitterrific is that I can be anywhere, shopping, getting a haircut, or at a conference and have the ability to take a cell phone picture for upload to Twitter with a brief description about what's going on.

Twitter and Facebook are such interesting places to socialize online while sharing information that I've neglected my blog for quite some time. To address that problem, I'll be posting summaries of recent Tweets to my blog here where you can catch up on some of the things I've been talking about. Although, this is a bit like sitting next to someone that's on the phone since my Tweets here are only half the conversation some time. To get the full benefit of Twitter you really have to follow both sides of the conversation.

To accomplish updating this blog with my Tweets, I whipped up a quick ColdFusion script to pull in my Twitter RSS feed and convert to HTML with all usernames and URLs converted to actual links. I'll next try to automate this so that my blog gets regularly updated with my Twitter summaries.

Of course, if you're already on Twitter, Follow Me, and on Facebook Add Me as a Friend.

How to read tweets

  • If a tweet begins with an at sign, its a username, like @JoeSchmoe. Its at the beginning because I'm replying to something that Joe Schmoe tweeted earlier.
  • If a @username appears later in the tweet but not at the beginning, then i'm refering to that user, like Hey, did you see that cool widget that @JoeSchmoe built?
  • If a tweet begins with RT, that means Re-Tweet, and its kinda like holding up a megaphone. On Twitter you Follow or subscribe to certain people, and in turn other people follow you. The people that subscribe to your tweets may not be following the people that you subscribe to. So by retweeting someone that you subscribe to, you are amplifying that persons tweet by broadcasting to all the people that subscribe to you. Confused yet? If your subscribers are interested in the retweet, they may then follow that original person (a.k.a tweep or twit).
  • If a tweet has a word beginning with a pound sign like #photog, that's called a hashtag. Hashtags are used as labels to identify a topic for your tweet. Later people can search by hashtags to see tweets from everyone that talked about that topic. In this case, #photog means the tweet is about photography or a photographer. This in contrast to #photo without the last 'g' and it is used for tweets about a particular photo.


Recent Tweets for Mon June 29, 2009



Mon Jun 29 10:09 PM
2 nights in a row my son doesn't want to sleep coz he's not feeling well. Long nights of frequent rocking, not much sleep for me.
Mon Jun 29 4:45 PM
@JoshuaCurtiss Pretty sure it was CF5 -> CF6 w/ the J2EE conversion. I was on the team at Macromedia that reviewd poss bkwds compat issues
Mon Jun 29 4:39 PM
@jlamoree Holy cow, a fradulent Lactating Lesbians infringement claim from Cream Ridge, NJ. That just, em, *sucks*!
Mon Jun 29 12:52 PM
@jonbcampos Apache XSL-FO http://bit.ly/Xrhar


[More]

Perspective on ColdFusion's Big Question (TM)

Just wanted to share a reply I made on GetSatisfaction to provide a historical perspective to the question "What really is the future of ColdFusion?". Before you ask what the future holds, its good to look back to see where ColdFusion has been since its inception in 1995.



CFMX 7 (released Feb 2005) was the release where product adoption saw the first major boost since the "MX" overhaul. Since CFMX 6 (released June 2002, in a down economy) was a re-architecture in Java/J2EE from the earlier CF5 (released May 2001) written in C++, there were few new features introduced and there was an associated learning curve now that the product had a Java foundation.

Problems in the re-architecture surfaced, slowing new adoption of CFMX6, leading to the point release 6.1 (released July 2003) which for the most part corrected all the issues and restored the waning product reputation.

ColdFusion MX 7 was a feature rich release, which attracted many new developers, most of whom had begun to grok CFCs and Java integration. The post 9/11 economy had generally recovered as well, adding to an increase in technology spending.

With most product release cycles, there's a decline in sales or tail at the end, and ColdFusion 8 (released August 2007) saw another major boost in adoption over the tail as it too was a feature rich release that provided solutions to many contemporary problems in Web Dev.

Frankly, IMO, nearly all negative connotations (i.e. "Legacy Software") about the ColdFusion Web Application Server are due to anachronistic experiences with earlier versions of the product in the mid/late 90's. Those opinions seem to be expressed from developers that are less familiar with the revisions and enhancements found in recent ColdFusion versions. (Case in point)

[Added note: The easy learning curve, weak typing, and case-insensitivity in the product are among some factors that may have been conducive to poor programming practices... i.e. give them enough rope to hang themselves, so to speak. Does anyone remember memory corruption from not locking shared scope variables? That whole conundrum went away with CFMX]

Personally, I think ColdFusion is a fantastic product and I love using it. It has an extensive, contemporary tag library on a stable Java base and Web application development time can be short and sweet due to its perpetual focus on RAD.

ColdFusion 9 is well known to be underway and will further address solutions to where technology is going. Furthermore, risk due to proprietary software is mitigated by the release of third party CFML engines which can provide a core of language features if not the full, rich diversity of language found in Adobe's product.


To throw in a plug for myself, I'm currently seeking full time, permanent employment in the greater Boston area. See: Adobe Expert Seeking ColdFusion / Flex Dev or QA

View Steven Erat's profile on LinkedIn

Power Mac G5: The lights are on but no one's home

Power Mac G5 OS X 10.4, 30The recent ice storm that crippled New England and left me looking for other shelter seemed to have killed my 2005 Power Mac G5. The night of the ice storm the lights in the house blinked off, then on, then off again... They stayed off for four days, along with the furnace, until the power company put my town back on line.

A week later, when I attempted to start the G5 in my home office the main power light came on, and stayed on when ever it was plugged in, and the desktop would not begin the boot sequence, nor were any sounds emitted such as the fans starting up.

With lots of other things to take care of I delayed troubleshooting it, but thought that it would eventually require a trip to the Apple Store Genius Bar for a drop off. Today I finally did a search and came up with this little wonder:

How to reset the SMU on a Power Mac G5

The SMU (System Management Unit) is a microcontroller chip on the logic board that controls all power functions for your computer. If your computer is experiencing any issues regarding these functions, resetting the SMU may resolve the issue. The functions controlled by the SMU include:

* It tells the computer when to turn on, turn off, sleep, wake, idle, and so on.
* It handles system resets from various commands.
* It controls the fan.



The guts circuit board did not look the same as in the photo on their article, so I started pushing all the transistors and every little silvery bumps I could find. Close to my RAM memory slots, I finally found a tiny, round, silver button on a small black square base that was in fact slightly pushable.

Upon reassembling the tower's side door and re-inserting all the cables, I was very happy to hear that little whir of the fan as the computer took its first breath after being resuscitated. The prospect of waiting in line at the Apple Store during Christmas week was something I was not looking forward to.

My Application to CDIABU in Retrospect

I was searching through my email for something today, and I came across my application to CDIA where I wrote this in the Personal Objective section almost two years ago.

Why have you chosen to pursue an education in the field of Digital Imaging Arts?
Through achievement, motivation, and serendipity I've been fortunate to have earned an esteemed position as software quality assurance engineer at Adobe Systems. Within this vibrant, professional atmosphere my personal growth as a serious amateur photographer has accelerated significantly.

At this nexus, I wish to seize the opportunity to expand my skills horizontally by bridging my experience in rich internet development with the digital imaging arts and professional photography.


What do you hope to achieve by enrolling in the program at the Center for Digital Imaging Arts?


With the completion of a certificate in Professional Photography I will have formalized a long standing, deep, personal ambition which will allow me to thoroughly exercise my passion with technical excellence.



Further, I strive to achieve a role at Adobe where I can consult with, collaborate with, and guide Adobe customers towards their own creative endeavors and solutions. Knowledgeable in the diverse array of photography, digital imaging, and next generation web application development I will be able to translate my success into the success of my customers, my colleagues, and my company.

To Flex Camp, and Beyond!

A week from today will be the 2nd annual Flex Camp Boston at Bentley University. At a very modest cost, this is a full day event packed with sessions at the intermediate to advanced level given by industry experts. Register for Flex Camp Boston.

For the last year I've been on the Flex SDK team as a Quality Assurance Engineer, and before that I had excellent run of more than 7 years testing and supporting ColdFusion. I know most of the speakers that will be presenting at Flex Camp and can attest to their passion for building the next wave of Rich Internet Applications, so I fully encourage you to attend if you haven't signed up yet to share in the excitement and mingle with your peers.

This will be an unexpected reunion of sorts for me as I suddenly find myself as a customer rather than employee. With the extra time as I seek new employment I'll immerse myself in training with Flex and AIR, and try to produce an application as an online reference to demonstrate as an example. The odd thing about QA'ing a software product is that you are exposed to narrow facets in which you dive very deeply, and don't often get the chance to practice the breadth of the product. My success in ColdFusion QA was largely dependent on the many preceding years where I provided "gold" level support for the product, something which required me to constantly explore and exercise every nook and cranny of the CF app server and language.

My first inclination for a Flex app is to build my own photography business website in Flex to avoid the cost of purchasing one of the reputable but expensive prebuilt websites from places like LiveBooks, BigFolio, or A Photo Folio.

Finally, I'd like to thank everyone from coworkers to customers to local cfug friends for taking a moment to contact me and express their thoughts and show their concern. People have been writing and chatting intensely while offering job tips and advice. As I mentioned on Facebook, I've never before felt the online community to be as tangible and real as I do now. Thank you all, and I hope to see those of you in the area at Flex Camp!

More Entries