Online PHP Training w/ Edward Tanguay
I haven’t posted any recommendations for products on this blog in a long time but I wanted to be sure to mention an online PHP Training course that I was shown today by Edward Tanguay.
I was first introduced to Edward Tanguay by his brother, Pete Tanguay, CEO of Rock-Pond Solutions. In addition to being good friends, Pete and I have several things going on together and whenever we get together to discuss these projects, the topic of web development always comes up.
Pete is always telling me about new and interesting projects that his brother is working on in Germany so when I discovered Edward on Twitter, I decided to follow him, his twitter name is @edward_tanguay. This evening Pete sent me a link to some online training videos that Edward had released that I want to recommend to my web developer readers. Here’s some info:
PHP 5.3: Advanced Web Application Programming
Learn the nuts and bolts of advanced web application programming! Join Edward Tanguay as he dives into PHP programming. In his workshop, you will learn that it only takes a few minutes to get an Apache web server, a MySQL database, PHP 5.3, and the Eclipse editor up and running using the free XAMPP installation tool. Edward will then cover the PHP syntax, such as operators, loops and variables, before moving on to make you a PHP expert, tackling object orientation, databases, XML parsing, web services, security and much more. The training also includes important new features of PHP 5.3, such as namespaces, lambdas, and late static binding. Once you complete this video training, you will have the skills you need to build next-generation dynamic websites.
If you are interested in purchasing this online training video for $39.95, you may do so by visiting: http://www.video2brain.com/en/products-17.htm#/content?t=2. There are also some preview chapters to the series available at this link. As a programmer I can appreciate the clarity and thought that went into putting together these videos. Here’s a little bit of information about Edward if you are interested:
Edward Tanguay: Website & Software Developer, Instructor
Edward Tanguay grew up in Colorado, U.S.A., where he wrote his first computer program on an Apple II Plus in 1983. Shortly after that, he and his brother founded a software startup, where Edward programmed in dBase II and Clipper for clients in the hospital and investment bank industries. When their successful company was acquired by a consulting firm, Edward decided to pursue his passions and dreams, i.e. study philosophy, learn foreign languages, and live and work in Europe!
Thanks to the excitement of the World Wide Web, Edward got back into programming in the 1990s, this time focusing on database-driven web applications based on CGI/Perl, HTML and JavaScript. Just as the dot-com boom got underway, he began working for startup companies in Berlin, where he improved and adapted his programming skills to the web: PHP/MySQL, ASP/Access and later XML/XSLT. At the same time, he held IT workshops at the Berlin-based Humboldt University, Free University and Technical University and wrote online articles on upcoming technology for webdevelopersjournal.com.
In 2002, he worked as a consultant for Microsoft Germany and gained experience working in large teams on enterprise projects, where he acquired knowledge of the .NET platform, C#, SQL Server as well as the intricacies of object-oriented programming and design.
He is currently working as a .NET Developer in Berlin and juggling numerous PHP projects, including his own open-source PHP framework named Datapod. http://www.tanguay.info
EP:002 – The Cotton Club Podcast
I am still continuing to get my audio settings tweaked for these shows so please bear with me on this, hopefully in the next few episodes I can get this all ironed out. I wanted to release another edition of The Cotton Club this week before heading out of town so I asked my good friend Bill Wheeler of SWScripts to be on the show, Bill is also @swscripts on Twitter.
Some of the topics that Bill and I discussed on this weeks show include:
- On-Air Bids (PHP based software application written by Bill and used by radio stations and media outlets to manage live auctions).
- MODx CMS / PHP Framework
- Pleth’s new entry-level CMS solution for Non-Profits, Churches.
- PHP IDE’s / Programming Environments
- Iphone Applications (Active Sonar, Waze, and News Stand)
Some Personal, Business, & Development Goals for 2010
The other day I was looking over my personal goals that I had set going into 2009. I feel really good about what I was able to accomplish in 2009 but unfortunately there are still a few things that I am going to carry over into 2010.
This is not to say that 2009 was a bad year, in fact I would say that 2009 was probably one of the best years I have had in a long time, especially in terms of business and personal development.
Personal Goals for 2010
I have set some pretty high goals for myself personally going into 2010 as compared to what I set for 2009. I have learned that I respond well when I put pressure on myself. Hopefully by setting the bar high I will be able to motivate myself to another level personally.
- Private Pilots License: One of the goals that I had for myself coming into 2009 was to get my pilots license. While I did manage to get a few lessons in I didn’t really devote a lot of time to accomplishing this goal. I am going to try to put forth a little more effort this year to get my license. I have checked into some classes offered by UCA for private pilot certification.
- Business Networking: One of the things I am most proud of in 2009 was helping to organize the Central Arkansas Refresh user group. I reflected on the #Refresh group a few weeks ago and gave some insight into how it all came together. Basically social media powered the group, but the group itself and our monthly meetings allowed me to make some awesome business contacts in the Central Arkansas area. I hope to continue growing my personal network in 2010 through outlets like the #RefreshCA group as well as attending various conferences.
- Speaking: I was fortunate enough to have been asked to speak at couple of events this past year on a variety of topics, primarily interactive marketing and social media. I really enjoyed being a part of these events and I hope to continue to take advantage of these opportunities when they present themselves.
- Diversify: While my passion is, and always will be, the internet, I plan on becoming involved in a few areas outside the web in 2010 as well. I have a lot of irons in the fire right now and I am not ready to make any announcements regarding any projects or anything just yet but I can assure you this, whatever business deals I get involved in 2010, they will strictly be sideline projects. I plan on staying actively involved doing what I am doing until I retire, these side projects will just hopefully help me retire a little bit sooner.
- Run in a 5k: Another one of my goals coming into 2009 was to become more healthy but I obviously didn’t do as well in that department as I would have liked. Hopefully I can do a lot better job in 2010. In order to push myself toward this goal I am going to try to run in at least one 5k this year. I haven’t decided on where or when yet, but after the first of the year I am going to start running and try to make this happen by at least the end of the summer.
- Build my Personal Brand: In a lot of ways I have already been building my personal brand by blogging here at The Cotton Club and by staying pretty active on social networks like Twitter, Facebook, and LinkedIn, but I am going to try to take things to another level in 2010 by adding some exciting new things. One of the things that I will be adding is a podcast. I haven’t worked on the specifics just yet as to how often it’s going to be released or even when the first episode will air but I can assure you that it will happen in the first quarter of 2010.
Business Goals for 2010
My partners and I had a great year in 2009 despite the economic downturn that plagued the nation. We managed to pick up some great new accounts as well as further refine the focus of our business. I would like to build upon some of the products and services that we had a lot of success with this past year. Here’s the list of business goals I have put together so far…
- Email Outsourcing / Archiving: I have blogged a lot the past few months about our premium email products and archival solutions because a lot of attention has been placed on regulatory compliance in a lot of industries. I think that we have a very competitive solution that will continue to do well for us in 2010.
- Shared & VPS Solutions: Over the years my partners and I have made a tremendous investment in our hosting infrastructure. Today, thanks to virtualization, we are able to provide VPS, or virtual private server, solutions to development firms for a very competitive rate, even on a national scale. I am going to try to make an effort to promote our hosting solutions a lot more in 2010 than I did in 2009.
- Ministry Web Solutions: This past year we launched Powersite ministry solutions to showcase our web design and development packages geared toward churches or non-profit organizations. We have had a great response so far from the Powersite launch and I hope to continue that momentum going into 2010.
- Social Media / Interactive Marketing: My partners and I had a lot of success this year steering a lot of our clients into the realm of social media. We also saw some tremendous results from our efforts. I hope to make social media an even-more integral part of what we provide for our clients in 2010.
Development Goals for 2010
I didn’t do a lot of custom application development this past year so I have come up with some goals for the upcoming year to get me back into the development mindset. Here’s what I have come up with so far…
- Develop 2 iPhone Apps: I have two ideas that I think could be profitable, I just have to finalize my project parameters and complete the markup models for each project, which I should be able to do by the first quarter of next year, then I have the rest of the year to familiarize myself with Objective C programming and knock these projects out.
- Develop 2 Wordpress Plugins: I also have two ideas for Wordpress plugins that I want to release in 2010. I already have started coding on both of these projects but haven’t had the spare time to devote to wrapping them up. I typically code on these types of projects after hours in between client projects. Hopefully I will find the time in 2010, stay tuned to this blog for info on both of these plugins.
- Develop 2 Web Applications: I also have two web applications, both geared toward the agriculture industry, that I should be able to launch in 2010. Both applications are 75% complete, I am just waiting on some regulatory guidelines to be handed down by a few more states and then I can wrap everything up. I won’t go into a lot of detail about these apps right now but will when we finally roll them out.
- Develop a MODx Project: My business partner Greg primarily works in MODx while I develop most of my projects in Wordpress. While both solutions are great for content management, they are both very different in terms of their core foundation. I would like to develop a project this next year in MODx so I can become more familiar with it’s operation and functionality.
Current Projects: Custom PHP/MySQL Members CMS
It’s been a long time since I actually coded a custom standalone php application for a client and I found my skills to be a little rusty at first. The reason I don’t get to code a lot of custom applications anymore is because most of the client projects we see today are built on top of a cms like wordpress or a framework like modx. In this instance the client already had their website developed internally and just needed us to develop an enhanced members area so they could communicate and share information with their members.
The membership table for this project has approximately 500 members in it, each member has the ability to authenticate by login and then view protected content. Members can also submit changes to their contact or login information that has to be approved before it’s accepted. Of course the normal features such as lost password restoration are also in place.
This solution was built entirely using PHP and all of the data is stored in 17 tables inside a MySQL database. For easier management of a large membership database with over 500 members, I also built in the ability to export and import data to the database from either XLS or CSV format. This should be another convenient aspect of the solution for our client.
I am pretty anxious to demo this application and bug-test it this next week. I used the newest version of PHPRunner IDE to develop this project and even though I was a little out of practice I still was able to code the project a lot faster than if I had used another tool like Eclipse or Notepad. The new version of PHPR is very nice, it’s been a while since I upgraded and have to admit that there are so many new features that I haven’t even looked into yet.
Checkout the O’Reilly Answers Social Network
I am typically not one to recommend a social network to my clients w/ the exception, of course, of the already established networks such as Facebook, Twitter, LinkedIn, Tumblr, etc. My whole philosophy regarding social networking at this stage of the game is that if you want to build a community, build upon what’s already out there as much as possible (Facebook API, Twitter API) and don’t try to re-invent the wheel.
However, I have always felt that strong, reputable Niche networks could exist externally if they were packaged correctly. A good example of such a network that is currently in Beta right now is O’Reilly Answers. O’Reilly has a good vision with this network, and of course they have some really positive things in their favor already such as Awesome Reputation, an Established Following, and a Trusted Name, what more could you ask for?
You can also look at the Network and tell that there were some clear objectives put into place while developing this solution. They obviously wanted it to be user friendly, functional, and interactive. There are basically three ways you can interact inside O’Reilly Answers:
Share Your Knowledge
You can actually enter in blog posts and tag them according to your subject matter and other users can comment you on your posts. This is one area that I think they could have done a little differently. We all manage our own blogs externally, why would we want to post our content on their website as well and have 2 comment systems running at the same time to follow up w/ readers? My thoughts on this part are that they could have put into place some sort of RSS option where you could aggregate your content from your blog and have your friends inside the network click out to read your posts. Maybe I am not seeing the big picture on this, but as a workaround I posted about 3/4 of 2 blog posts into their network along w/ a link at the bottom to get to the remainder of my post on my blog.
Ask A Question
Since O’Reilly has been catering to the Technically Minded community for so long, you can rest assured that some pretty sharp folks are going to be hanging around in this community. Well, let’s say you have a question about a project you are working on, you can post that question to the community, tag it, and before you know it some of the sharpest minds in the world are answering your question. This is an area where I really see this network having value for a lot of us…
Answer a Question
Let’s say someone asks a question and you know the answer, you can provide them w/ your insight w/ ease. I think that this networking model is going to speed up the “obstacle to solution process” once the network get’s off the ground.
———————————————-
Also, just a few notes regarding the Network as it is right now. There are obviously some bugs, I had a couple of errors pop up on me when I was setting up my profile, this is to be expected w/ a new release like this. Give them time, I am sure that O’Reilly will iron out all of the kinks very soon. Here’s some basic information that was forwarded to me today from our Account Rep at O’Reilly…
We’re launching the beta of O’Reilly Answers, and I’m inviting you to be part of it. In brief, O’Reilly Answers is a community site for sharing knowledge, asking questions, and providing answers that brings together our customers, authors, editors, conference speakers, and Foo (Friends of O’Reilly).
Why Answers, and why now?
O’Reilly is at the center of an amazing exchange of knowledge sharing and idea generation. We’ve created the usual means of facilitating communication between customers, O’Reilly folks, and the outside experts we call "alpha geeks" who contribute to O’Reilly books, conferences, and websites. We can connect through reader reviews, errata submissions, book forums, blog comments, Get Satisfaction, our customer service department, and more. But too much of this conversation is siloed, and not enough is public (e.g., discussions on our internal mailing list for editors, or personal responses to customer questions). O’Reilly Answers will be the place where much of that communication happens from this point forward.Why participate?
The lofty reason: Like O’Reilly, you want to "change the world by spreading the knowledge of innovators." That’s our mission, and we’ve been fortunate enough to build a community of passionate, committed people who love to learn and share their knowledge as they work towards a better world for us all.The "nice, but what’s in it for me" reasons: reputation, recognition, and rewards.
Get Recognized: "Find interesting people" is a core activity at O’Reilly, and an important component of our success. We see Answers as an important way to discover and connect with our next authors, online instructors, videographers, and speakers.
Build Your Reputation: You’ve learned a lot, why not get credit for all that knowledge? As your submissions to Answers are voted up, your personal reputation on the site increases. At launch, your reputation will be based solely on your participation in O’Reilly Answers. Soon, we’re expanding across oreilly.com, so the book errata and book reviews you’ve submitted, books you’ve registered, and conferences you’ve attended, will add reputation points. You’ll also earn badges to mark accomplishments and milestones.
Earn Rewards: Glory is great, but discounts and deals are nice, too. We want to reward your contributions to the O’Reilly community. Shortly we’ll have a point-based system in place that you can redeem for books, training, courses, and conferences. Details soon, but in the meantime, any actions you take now will count towards your total points.
This is just v.1: The best part of any project on the web is watching it take on a life of its own. With that in mind, we’re looking forward to *your* suggestions about where O’Reilly Answers should go, what features should be added, and what benefits and rewards we can offer all of you.
I’d like to acknowledge the projects that have proceeded Answers and inspired us, such as SitePoint Forums (we distribute their books), StackOverflow, Yahoo! Answers, Knol, and many others. They’re great resources, and we think the O’Reilly community can create a useful site that’s, well, a different kind of animal.
One last thing: O’Reilly Answers is in beta and you may encounter bugs. We’re still working on many improvements to the site, such as feeds for each tag, but would love to hear your suggestions for features and improvements. Please send any suggestions/questions/bug reports to answers@oreilly.com.
Until next time–
Marsee Henon
Also, if you should signup, be sure to add me as a friend, http://people.oreilly.com/cotton
WordPress Exploit Scanner
If you are like me, you want to make sure that the software you run is as secure as possible. My partners and I even subscribe to several third party services that actually scan our servers looking for exploits that could be taken advantage of by hackers or script kiddies. Since one of the most frequent CMS solutions we use is WordPress, this plugin caught my attention.
The WordPress Exploit Scanner is a plugin that searches the files and database of your website for signs of suspicious activity. While it won’t stop someone hacking into your site, it may help you find any uploaded or compromised files left by the hacker on previous attempts. It can also help you identify any weaknesses that you might have so you can harden your installation.
Here’s a little bit more on the Exploit Scanner plugin if you are interested:
When a website is compromised, hackers leave behind scripts and modified content that can be found by manually searching through all the files on a site. Some of the methods used to hide their code or spam links are obvious, like using CSS to hide text, and we can search for those strings.
The database can also be used to hide content or be used to run code. Spam links are sometimes added to blog posts and comments. They’re hidden by CSS so visitors don’t see them, but search engines do. Recently, hackers took advantage of the WP plugin system to run their own malicious code. They uploaded files with the extensions of image files and added them to the list of active plugins. So, despite the fact that the file didn’t have a .php file extension, the code in them was still able to run!
You can download this plugin here: http://ocaoimh.ie/exploit-scanner/
MODx Revolution 2.0.0 Beta2 Released
My partner Greg will probably blog about this a lot more on his blog because he primarily handles all of our MODx projects as to where I handle most of our Wordpress projects. Not having worked with MODx very much I am still impressed with all that it brings to the table in terms of rapid deployment and content management.
If you are looking for a good enterprise level content management system there are several of them out there, Drupal, Joomla, and even Wordpress has enterprise level applications these days, but for us MODx has been a great platform for us to build upon. We recommend it highly whenever we talk with other developers and one of the things that we like about it the most is that it is supported by a highly active community of developers. The activity in the support forum at MODx is unlike anything you have ever seen tied to open source software.
If you are already familiar with MODx, here’s a short list of the improvements / new functionality that makes up Revolution 2.0.0.
- Manager upgraded to ExtJS 3.0 from 2.2; now faster and more responsive.
- You now can drag/drop Elements and Resources directly into any content field and watch as it builds the tag for you.
- We’ve added a Visual Tag Builder for Elements, where after drag/dropping them into fields, you can select a Property Set and customize values from a form, and watch it build the tag syntax for you after you’re done.
- Now you can Quick Update or Create any type of Element or Resource. Don’t want to leave a page to update another Document? Quick Update it!
- Toggle your Manager layout between tab-based and portal based with a setting.
- Speed improvements to the backend manager.
- Fixed issues with uninstalling packages; they properly revert and save zipped copies of older versions.
- Package Download section now grays out packages you’ve already downloaded.
- Manage all Plugins assigned to an Event now easily from an intuitive grid.
- Added new methods for zip compression for installation and packaging that speeds up install time.
- Many, many bug fixes across the framework
As soon as I get caught up on a few projects I have going on I plan on spending some time getting acquainted with it on one of our sandbox environments.
Update: This just goes to show the response time that all of the people surrounding the MODx project have regarding support, etc. I just posted this blog post on my blog and walked to the kitchen to get a bottle of water. When I returned I already had a comment waiting on me from Shaun with Collabpad. Here’s what he had to say regarding the beta 3 release:
You might want to check out 2.0-beta-3. There was an important security fix in beta3 that we recommend users upgrade to.
Thanks for the article! We’re glad you like MODx, and hope you enjoy the future of MODx in Revolution. Feel free to leave us feedback and suggestions for Revolution on our forums (they are super active! it’s crazy!)
Locking Down Authentication Inside PHPRunner
One of the biggest challenges you face when building hosted applications is how to prevent brute force or guessed password authentications. Especially given the number of warez type applications that are out there that allow unsavory users to do just that. Well, I found a resource on Xlinesoft’s website that demonstrates how to do block a user after three unsuccessful attempts to login to your application.
This schema uses visitors IP address to store log attempts in the database and block access to to the login feature for 30 minutes after the third unsuccessful attempt. This schema involves Events function which is available in ASPRunnerpro 6.0/PHPRunner 5.0, I have reposted the processes involved for PHPRunner below, but you can find the ASPRunner notes here…
Step One:
In MySQL Server run the following script to create table in your database that logs login attempts. The box below demonstrates the MySQL command.
1: CREATE TABLE `LoginAttempts`
2: (
3: `IP` VARCHAR(20) NOT NULL,
4: `Attempts` INT NOT NULL,
5: `LastLogin` DATETIME NOT NULL
6: )
Step Two:
Open your PHPRunner project and go to the security tab and switch on the “Create Login Page” checklist.
Check the Username and password from database option and choose appropriate fields. If you have no table in which all of the login details are stored you have to create it.
Step Three:
Add three global events on the Events tab: BeforeLogin, AfterSuccessfulLogin, AfterUnsuccessfulLogin. Below you will find the PHPRunner example for this:
1: <?
2: function BeforeLogin($username, $password)
3: {
4: //********** Custom code ************
5: // check if this IP address is currently blocked
6: global $conn;
7: $sql = "select Attempts, LastLogin from LoginAttempts where ip = '" . $_SERVER["REMOTE_ADDR"] . "'";
8: $rs = db_query($sql,$conn);
9: $data = db_fetch_array($rs);
10:
11: if (!$data || !strlen($data["LastLogin"]))
12: return true;
13:
14: $atime = db2time($data["LastLogin"]);
15: $time = mktime($atime[3],$atime[4],$atime[5],$atime[1],$atime[2],$atime[0]);
16: $diff = (time()-$time)/60;
17:
18: if ($data["Attempts"]>=3)
19: {
20: if($diff<30)
21: {
22: echo "<p align=center><br><font color=red><b>Access denied for 30 minutes</b> <font></p>";
23: return false;
24: }
25: else
26: {
27: db_exec("update LoginAttempts set Attempts=0 where ip = '" . $_SERVER["REMOTE_ADDR"] . "'",$conn);
28: return true;
29: }
30: }
31: return true;
32: }
33:
34: function AfterSuccessfulLogin()
35: {
36: //********** Custom code ************
37: // clear previous attempts
38:
39: global $conn;
40: db_exec("update LoginAttempts set Attempts=0 where ip = '" . $_SERVER["REMOTE_ADDR"] . "'",$conn);
41:
42: }
43:
44: function AfterUnsuccessfulLogin()
45: //********** Custom code ************
46: // increase number of attempts
47: // set last login attempt timeif required
48: {
49: global $conn;
50: $sql = "select * from LoginAttempts where ip = '" . $_SERVER["REMOTE_ADDR"] . "'";
51: $rs = db_query($sql,$conn);
52: $data = db_fetch_array($rs);
53:
54: if($data)
55: {
56: $attempts = $data["Attempts"]+1;
57:
58: if($attempts==3)
59: db_exec("update LoginAttempts set Attempts=" . $attempts . ", LastLogin=now() where ip = '" .$_SERVER["REMOTE_ADDR"] . "'",$conn);
60: else
61: db_exec("update LoginAttempts set Attempts=" . $attempts . " where ip = '" .$_SERVER["REMOTE_ADDR"] . "'",$conn);
62: }
63: else
64: db_exec("insert into LoginAttempts (Attempts,IP,LastLogin) values (1, '".$_SERVER["REMOTE_ADDR"] . "',NOW())",$conn);
65: }
66: ?>
Step Four:
You should finish the code generation / compiling process and upload your application. It’s important to remember that by doing this, your visitors have to enter their username and password to gain access to the site. After the third unsuccessful login attempt, their IP addresses access will be denied for 30 minutes. When the visitor tries to login when the account is blocked they will see message saying access is denied.
Find out how to do this for ASPRunner also…
——————————————————————
There are a lot of other useful resources outlined for PHPRunner users in the Articles section on Xlinesoft’s website, you can find them here…
Facebook | PHPRunner User Group
I recently started a PHPRunner user group on Facebook in hopes of connecting with other PHPR programmers. So far we have 6 members, myself and of course Sergey, who is the CEO of the company that puts out PHPRunner, and four other guys. My main goal for creating the group was to provide some awareness of PHPRunner as well as being a way for PHPR developers to reach out to one another for paid assistance on projects built with PHPR. Honestly there have been times that I would have hired a developer to assist me on projects if he was familiar with PHPR.
Of course the formation of the group was met with a little opposition by users on the support forums because they didn’t see the need for such a group because the forums on xlinesoft’s website are already pretty active. I pointed out that there are user groups on Facebook for other IDE software clients such as CodeSmith, CodeCharge, and Eclipse and having a group for PHPR would probable help bring exposure to the product.
I also have always felt funny about soliciting developers from message boards, not to mention it’s forbidden on a lot of support forums, but having a Facebook group is perfect for something like this. I have made it very clear that the user group is not a support forum and is only intended for developers to network with one another, showcase their work, and to post projects for bid. I am hopeful that the group will grow in size and we can develop a pretty good community but I am also a realist and I know that PHPR users make up a pretty small piece of the pie. As I am sure a lot of you that have talked with me about PHPR already know, I am pretty hung up on this code generation thing and out of all the products I have seen on the market, PHPRunner is the best one out there.
If you are looking to learn more about PHPRunner, be sure to checkout their website.
Facebook | PHPRunner User Group
Critch on VMware, Apache, PHP/MySQL
I am happy (bordering on giddy) that our server engineer / administrator Matt Critcher is now blogging, dude is probably one of the sharpest guys I have ever met and he is an all around cool guy to hang with too, but beware of the fancy cheese he brings to dinner parties because you could find yourself in the emergency room on New Years Eve thanks to a long-standing penicillin allergy.
As some of you might know we made the transition to Virtualization a while back and have been extremely happy with the versatility it has brought us with our managed hosting and vps products that it has allowed us to bring to our clients, but with growth there can also be growing pains, it is for this reason that I am so glad we have Matt in our corner, dude knows his stuff and he can get to the bottom of an issue better than anyone I have ever worked with.
Lately we have been transitioning to VMware and have had some issues w/ websites that are slow to respond via browsers, but yet they still ping out okay. It’s been a weird week or so, here’s a post that Matt put together the other night about the issues, I thought maybe someone else could benefit from his findings down the road:
I posted a few weeks back that Pleth had transitioned some of their equipment over to VMware Server and for the most part it’s been a very smooth process. But, as of late we’ve ran into some slowdowns, especially on the VPS with Plesk (which happens to host several of our websites). After doing a bunch of research and spending many a late hour digging through tons of mpstat and other sysutils data I think I found the culprit(s).
VMware Server, unlike the ESX/ESXi products, does not run in a Type 1 Hypervisor. This means that the underlying OS (in our case Red Hat Enterprise Linuxwas tuned out of the box for a general all-purpose server. This configuration isn’t always optimal for a Type 2 Hypervisor. It works just fine as long as things are "normal," but as the new VMware server got a larger load (in terms of I/O and CPU) performance went downhill.
One of the major problems has to do with how VMware Server uses disk-backed memory files (*.vmem). There is great debate on the web whether or not you should disable them, but one thing that is clear — when a site is busy, the file will be updated with memory information to reflect the changing memory of the VPS in question. This is where the problem lies — servers with unga-bunga hardware RAID solutions with 15K RPM disks and tons of spindles have a less of a problem with it but moderate quad-core Xeon and SAS disks in a RAID1 configuration like we and most other webhosts our size have it is a bigger issue. All those writes causes a wait-state in the CPU and therefore a backlog of transactions to be processed causing said server slowdown.
One way to deal with this is to modify the /etc/sysctl.conf to add (or modify) the following parameters:
vm.dirty_background_ratio
vm.dirty_ratioI set my vm.dirty_background_ratio = 2 and vm.dirty_ratio = 85
Basically what these 2 parameters do is dictate the percentage of memory that can be "dirty" before it begins to flush (background_ratio) and the percentage of memory that can be "dirty" before a forced flush begins. When these files are updated, we can either have them done in the background (hence the low number for background ratio) with pdflush which allows other processes to continue to run, or we can have them queued up and wait for a synchronous (forced) write causing the iowait states (hence the large number for dirty_ratio). The big gap between background writes and synchronous is to try to keep the background writes coming consistently and avoid the synchronous writes as much as possible. You’ll have to play around with these figures to see what works best for you. See this page about half-way down for a little more in-depth explanation of these two parameters.
I also made some configuration changes to PHP and Apache to try to get a tad bit more performance out of each of them. I had written out a whole list of stuff that I’d modified to post here, and as I was looking for websites to help explain the modifications, I stumbled upon this website from IBM that lists pretty much every change that I made to Apache and PHP.
If you want to tune your MySQL database, this website is invaluable. It explains almost every parameter that you can possibly adjust and how to adjust them. One that it doesn’t really get into though is
innodb_flush_log_at_trx_commit
Setting this to "2" will force the system to write out any changes to the transaction log when the commit occurs but will only cause a flush of this data from memory to disk once every second (which gets stuck in the scheduler and is handled in the background by pdflush). The default setting of "1" will write out to file and flush this data from memory every time a commit happens. On really busy servers with InnoDB tables, this can cause slowdowns if your server really isn’t designed to handle a heavy DB load (most webservers aren’t). The drawback to this is that if the system crashes, you could lose 1 second of writes. Depending on what you are doing, this might be acceptable. Setting this to 0 will cause the write every second, but if the server crashes you might lose a ton of data because nothing is done at transaction commit. Scary, but fast (to me, scary outweighs speed in this case).
None of these changes should be taken without first thinking about what might happen. We have a test box in our office that basically mirrors our production server that I could test on beforehand. The Apache and PHP config changes are easy — no server reboots required, and you’ll know almost immediately if you mess them up. If you modify sysctl.conf incorrectly, the server might not boot. Better test a few things out (a VMware VM is a perfect testbed for these settings) BEFORE you have downtime.
VMware, Apache, MySQL, and PHP Performance Tuning | www.mcritch.com
