We have a number of major releases planned for the beginning of 2010 for our server monitoring service, Server Density; but for now we have just pushed out a few minor updates – subtle changes make all the difference!
Disk usage alerts on any mount point
Previously we only alerted you about % disk usage on your root mount point. Now you can configure alerts on any point. This is useful if you have different partitions set up for databases, home directories, etc.
Alert suggestions
Next year, Server Density will start to be more proactive in helping you figure out what to do with the data we provide and making suggestions based on the data we have. The first part of this is alert suggestions, which are now available to all paid and trial users.
You’ll find the link to “Suggested alerts” for each server on the “Alerts” tab, which you’ll also notice has been slightly updated to work better if you have multiple servers.
It is important to remember that whilst we try our best and we use your server specs to customise the suggestions, you are still the best person to set alerts based on your knowledge of your server setup, the tasks it performs and previous experience.
Don’t forget to read our CPU load and memory usage guides.
UK VAT changes
A reminder that UK VAT reverts to 17.5% on 1st Jan 2010. Our prices are excluding VAT so only customers in the UK or EEC (who are not businesses) will be affected by this.
Support availability
Support will be closed on Dec 24th, 25th, 26th, 27th, 31st and Jan 1st. We will of course be monitoring things for emergencies. Let us know if you have any problems.
Merry Christmas!
Which leaves me to say, Merry Christmas! We’re looking forward to 2010!
A recent update to Server Density, our server monitoring service, replaces a free form text field with a drop menu when selecting your country dialing code. This reduces the possibility for error but was a real pain to implement because I couldn’t find a list of countries in drop menu form.
As such, I had to make it manually. And to save anyone else the time, here is the HTML and PHP code used to handle the menu.
Country dialing code HTML
Some countries have the same country dialing code (e.g. Canada and USA) so using that as the option value does not work. As such, I simply created a PHP array of all the countries – view on Github. This creates the HTML select drop menu – view on Github.
PHP processing
As the select drop menu uses a simple incrementing key, this needs to be translated to the correct value when the form is submitted. This is done using the previous array and a switch block – view on Github.
We have just started using Campfire internally so we can all chat as a group rather than individually over IM. Back in 2006 37signals posted how they were using an unofficial bot to post Subversion source control notices to the chat room and a Ruby library was released, but these are now either unavailable, unmaintained, old or simply don’t work.
Our products are developed in PHP and Python. I have used Ruby (on Rails) in the past but my preference was to use PHP to write a simple script that would pull in the latest entries from our source control activity RSS and “say” any new items in our Campfire chat room.
There is a very simple, official Campfire API that allows you to read and write to specific chat rooms. This is how you can use that API with PHP and Curl to post to your chat rooms.
Requirements
- PHP with Curl
- A Campfire user with an authentication token. You can create a separate user just for this or use your own, it doesn’t matter. The API token is from the “My Info” link in the top right corner.
The request
You can send your request using XML or JSON. I think JSON is easier to work with so used that, but the payload being sent is very small so it really doesn’t make much difference. It is literally just providing the body of the text you want to say:
{"message":{"body":"Hello"}}
Since it is so simple, I didn’t bother making it using json_encode(), but you could do this to save you having to handle escaping slashes.
If you use XML instead then make sure you specify this in the content type header below i.e. Content-type: application/xml instead of Content-type: application/json.
Authentication
You just need to authenticate using your token, within a CURLOPT_USERPWD curl option. No password is needed so we can just provide any value.
curl_setopt($ch, CURLOPT_USERPWD, 'authTokenHere:X');
so if your auth token was 27a3e5e8c1a4e75fbd0bed1465e413e4ea9b5c9c you would use:
curl_setopt($ch, CURLOPT_USERPWD, '27a3e5e8c1a4e75fbd0bed1465e413e4ea9b5c9c');
Executing the request
You do the post against your own unique room URL e.g.
http://boxedice.campfirenow.com/room/12345/speak.json
whichb you get from your browser when you click to enter a room. And put it all together with the following curl code:
$line = 'Lyra and her daemon moved through the darkening Hall, taking care to keep to one side, out of sight of the kitchen.';
$request = '{"message":{"body":"' . addslashes($line) . '"}}';
$ch = curl_init('http://boxedice.campfirenow.com/room/12345/speak.json');
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_USERPWD, 'authTokenHere:X');
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-type: application/json'));
curl_setopt($ch, CURLOPT_POSTFIELDS, $request);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$result = curl_exec($ch);
curl_close($ch);
Response
The response to a succesful write operation is the status code “201 Created”. We also include the complete XML or JSON for the final resource in the response. This is because you can usually get away with creating a new resource with less than all its regular attributes.
Handling errors is through standard HTTP error codes – see their API docs for details.
We have just pushed out a minor bug fix release of the Server Density monitoring agent, sd-agent. Accompanying this release are packages for Debian/Ubuntu and Red Hat/Fedora compatible systems so you can use the built-in OS package management provided through the likes of apt-get and yum.
sd-agent 1.4.1 includes number of bug fixes and tweaks and will now report back data immediately on start, instead of waiting 60 seconds. It has been tested on all our standard platforms with the addition of CentOS 5.4 x86_64, Fedora Core 11 Base x86_64 and Ubuntu 9.10 (karmic) x86_64 to the test suite.
Instructions for updating
Instructions for using Debian/Ubuntu package management
Instructions for using Red Hat/Fedora package management
Give us a shout if you have any questions or problems.
Everyone knows that problems always seem to happen when you are asleep, on holiday or away from your computer. Given that many people will be taking time off work for the Christmas holidays, just like when you go on a road trip and check water, oil and tyre pressure, here’s a short list of things check on your server(s) so you can relax and enjoy the warmth of a laptop fire without annoying down alerts.
Make sure you have sufficient disk space
If you’re using your servers in a steady and consistent manner, it should be straightforward to work out how much disk space you are using on a daily basis and plot that out to ensure you won’t run out on New Years Eve!
Check your /tmp partition
Following from the above, check that your /tmp partition has not filled up and has plenty of space free. The OS should manage this for you but there are possibilities where it will quickly become full (e.g. temporary log files).
Check log file sizes
System logs in /var/log should rotate but you should consider clearing some of the older files out if you no longer need them. This will help free up space.
Don’t forget log files that don’t rotate themselves like Apache access/error logs. These can become very big very quickly.
Dry run your backups
You don’t want to be messing with restoring broken backups on Christmas Day – do a dry run of your latest and archived backups to ensure they are working.
Tidy up any “hacked” work
Last night we experienced an issue with our server timezones, which were all reset when an automated OS update ran, because they had been set using a symlink (as recommended by many blogs). Make sure you use the official methods of setting things (i.e. not just symbolic links) so that if an update runs in the meantime, it doesn’t break it.
Check your update exclude files
If you use an automated update system like yum, make sure you have correctly define your excludes. This will ensure that critical parts of your server (e.g. the kernel, Apache or PHP) don’t automatically update themselves and break your systems.
Check your alerting
If you use something like Pingdom to alert when your sites are down, make sure they are configured properly. Ensure the “check for string” matches what you expect and look at the alert contacts to be sure that the right people get notified. Same goes if you are using our server monitoring tool, Server Density.
Run a reboot test
If you have full redundancy/failover, test a server reboot to a) check that the failover works and b) your server(s) come back up from a reboot.
Check your secondary machine
If you’re taking a laptop with you on holiday, make sure you can connect to your servers – do you have the latest VPN client? Do you have your SSH keys installed?
And when all else fails and you get that phone alert at 4am, don’t forget the list of dumb things to check and take your time – everyone else is on holiday too so nobody will actually notice*!!
* This is obviously a joke. I know I hate it when I see my uptime drop below 100%, even if nobody else cares!
Whilst everyone is aware of the role the law plays in daily life, it really only steps into view when something goes wrong. This could be a query from your local tax office or a dispute with an employee, but the most likely reason you would have any dealing with the law is with customers in relation to what your business does on a day to day basis. In a software as a service context, this will generally be in the realm of contract law, either with a custom contract for a specific deal/customer or through your general website/service terms and conditions
With a background in Law, I have an understanding of the need for tight legal documents when it comes to contracts. It is all very well copying a generic website T&C document but when it comes to actually relying on a specific clause, you want to make sure you’re fully covered. This is particularly important when it comes to the availability of software as a service, especially if it is providing critical infrastructure like our server monitoring product, Server Density, does.
Just like doctors, lawyers are expensive. To become a lawyer in the UK (and elsewhere) takes a long time and a lot of money. Not only do you usually have to complete a degree (law or otherwise), there are many years of professional qualifications before you can even start dealing with clients. Anyone who has dealt with lawyers will know that the hourly fee depends on the level of experience of the person – paralegal, associate and partner – and the experience really does make a difference.
And this is the paradox – you need to ensure you have all your bases covered before you start signing up customers (it’s awkward to change T&Cs after the fact), this is expensive and you won’t have the money to spend on legal fees until you have customers.
I never get the accountants in before I start up a business.
You could wing it and buy a cheap set of T&Cs from somewhere like Clickdocs. They won’t be custom, you won’t really understand what they mean (you have nobody to explain them to you) and if it ever comes down to relying on them, you better hope they were drafted correctly and you know which clauses to use. But they’re cheap and you can get all your customers to agree to a new set later on.
Or you could engage a lawyer to draft them for you. You will be able to spend time explaining your exact requirements to ensure that everything is covered, you can get explanations of what each term means and if you need to rely on them, you can have more faith that they’ll stand up in court.
For businesses that aren’t just an extremely simple website, I would always recommend the latter. It’s probably 10x more expensive than buying from the likes of Clickdocs but for £500-£800 you can get legal peace of mind. It’s a high cost, but only in the short term. If you’re running a long term business then that set of T&Cs will last and could save you £100,000s in legal fees should you ever have to go to court.
“Easy for funded companies, I don’t have the money!” you might exclaim. Boxed Ice is bootstrapped and I got our T&Cs drawn up for the launch of Server Density before I’d even applied to Seedcamp. Our lawyers (Orrick) understand this and deferred payment for 6 months. Good law firms look to the long term and by then you should be generating sufficient revenue to cover their fees.
“It’s fine, I’ll copy someone else and write them myself!” you might suggest. No. Only a fool represents themselves. T&Cs are completely ignored until it comes to rely on them, when they will be scrutinized to a level you wouldn’t believe. When that happens you want to be sure they are well drafted!
I have had an iPhone since February 2008, both with the original device and the 3G – I have not upgraded to the 3GS. During this time there have been regular updates to the OS, with perhaps the biggest being the launch of the App Store. I am very happy with my iPhone but I have always been aware of Android.
With the excitement around the release of Android 2.0 and the Droid handset in the US, two weeks ago I decided to purchase an Android phone to try alongside my iPhone. Many sysadmins and developers have iPhones, one of the reasons behind developing our own server monitoring iPhone application, but Android is becoming ever more popular as the platform develops and Apple fail to address the issues with the app store.
There are plenty of reviews comparing Android to iPhone generally so this post will focus specifically on the features sysadmins would find useful. I am a Linux sysadmin so there will be no comment in relation to OS X Server or Windows Server.
This is comparing an iPhone 3G running OS 3.1.2 (not jailbroken) against a T-Mobile Pulse running Android 1.5 (not rooted). I am going to be looking at core OS level features rather than the availability of “tools” apps such as monitoring or troubleshooting, discussion of which will be in a separate future post.
We use Google Apps for the @boxedice.com e-mail because of the integration into the other Google apps, the excellent spam protection and the Gmail web UI (particularly search). On my Mac I do not use Apple Mail.app but have a Fluid instance that accesses the Gmail interface.
The same applies on my iPhone – Apple Mail on the device does not work well with Gmail through IMAP because of the labels system. I have experienced problems when deleting and moving messages between what are “folders” on the iPhone, but also with the “All Mail” folder. As such, I use the mobile web UI through Mobile Safari. This uses the latest web technologies, like HTML5 and local storage, so that mail loads very quickly. However, I have experienced issues when I have intermittent connectivity – I travel by train quite often and sending mail breaks as the connection drops and comes back. This causes confusion as to whether a message has been sent or not because the UI is asynchronous and the action has sometimes not completed even though the UI indicates that it has.
iPhone Gmail Message List
iPhone Gmail Conversation View
Android includes a native Gmail app which syncs with my Google Apps account. It is tightly integrated into the OS and so includes support for push e-mail via the notifications panel built into the OS, something that is not possible with the iPhone web UI nor through the Apple Mail.app (which has to periodically pull mail). That said, it is not without its flaws and sometimes takes a long time to pick up new messages.
Android Gmail Message List
The conversations layout familiar to Gmail users works well and you can perform all the same actions such as search, archive, labelling, starring and deleting. I also pull mail in from a 3rd party POP account and the client is clever enough to use the right “from” address when replying to e-mails sent to different addresses.
Android Gmail Conversation
Conclusion: The iPhone web UI works very well but my preference is almost always for a local, native app because of the OS integration and being able to handle connectivity issues better. I do not use push e-mail because I do not want to be constantly interrupted but the integration into the Android notification system works well and is unobtrusive.
Chat
Following on from the use of Google Apps, we make extensive use of Google Talk for IM. This integrates very well into Gmail with the web UI having a chat widget, “buddy” status in e-mail conversations and searchable chat logging. That all works seemlessly on Android with the native client that runs in the background and again integrates into the Android notifications panel. If someone initiates a chat with me, it will pop up on Adium on my Macs, in the Gmail web UI and on my Android phone. I can choose which device to use and switch between them without interrupting the conversation.
Android Gtalk
This is not possible on the iPhone, which requires you to be viewing the web UI to chat. This is a problem with being unable to run background processes but regardless, there isn’t a native Gtalk app provided by Google. You would need to use a 3rd party app and probably make use of push notifications to get the chat alerts when it isn’t running.
iPhone Gtalk
We also use Skype and the experience on the iPhone is far superior than on Android because the iPhone has a fully native app that includes text and voice chat as well as both incoming and outgoing “landline” calls to your Skype number.
iPhone Skype
In contrast, Android has a “Skype Lite” app that lets you make and receive calls but through a local number that uses your bundled minutes instead rather than a direct VoIP P2P connection. Incoming calls work by setting up call forwarding to your mobile phone number, which uses your Skype credit. This works well when you are “mobile” as it does not require a data connection (the iPhone Skype client doesn’t work over 3G).
Android Skype
Conclusion: Since GTalk is built into the Android OS it is the clear winner here, particularly for the ability to seemlessly switch between devices. Skype is functional on both platforms but there are “hidden” costs involved with using Skype on Android.
Remote access – SSH
Server issues always occur when you’re not at your main computer so being able to connect remotely on your phone is very useful! Android has a free app called ConnectBot which looks like it could be a good SSH client but I was unable to figure out how to actually get it to connect! Having entered the hostname into the “quick connect” box there was no button to actually connect, nor could I add any “favourite” hosts. Entering an IP address was frustrating because the Android number panel returned to the keyboard after each “full stop” character, of which there are several in IP addresses!
Android ConnectBot Login
The ConnectBot app gives you access to the local terminal but attempting to execute the “ssh” command to connect to a remote host presented a permission denied error. Having connected to the local terminal, it was added as an automatic favourite so I tried to edit the details to allow me to connect to the remote host. Unfortunately, it didn’t seem to pick up the change in hostname and despite disconnecting and reconnecting, the client still connected to the local host.
Android Local Terminal
In contrast, there are several (free and non-free) SSH client apps available for the iPhone. Many are buggy and difficult to use but the one I have settled with is iSSH. This has almost all the features you’d expect from an SSH client – saved hosts, managing multiple connections, common shortcut keys. It also has a passcode screen to prevent access to the app.
iPhone iSSH
The key feature of any phone based SSH client is the ability to use SSH keys as they can be specific to each device and easily revoked. Both the iPhone and Android apps support these.
Conclusion: Since the Android app completely failed to work, the iPhone wins! There are several apps to choose from on the iPhone, free and paid, so you can pick which app suits you best. Android only has the one app.
Remote Access – VPN
None of our servers are accessible via SSH publicly – you must connect through our Cisco IPSec VPN. Support for this is built into the iPhone OS and is a simple case of flipping a switch in Settings to connect. Unfortunately, there is no such client in Android, nor any apps to allow connectivity to a Cisco VPN. As such, it is impossible to administer our servers via Android. To be fair, the iPhone OS only introduced Cisco VPN support in OS 3.0 and this is obviously very specific to our set up, but Cisco VPNs are used in many corporate environments and being unable to support them is a hinderance.
iPhone Cisco VPN
Conclusion: Cisco VPN only available on the iPhone.
General comparisons
Android Home Screen
iPhone OS is much more polished than Android. This is quite a subjective statement but the effort Apple has put into the interface shows. Usability is a key consideration when Apple set out their interface rules and that makes for a more consistent experience, which makes it easier to understand how to do specific tasks. Android runs on many different devices and so has to handle different button configurations (or lack thereof) which makes it difficult to discover how to do certain things like displaying context menus (of which there are none on the iPhone, except for copy/paste).
Whether this is the hardware or the OS I’m unsure, but I found Android to be less responsive. There are occasions when the iPhone lags for a few seconds but those are usually very rare. In contrast, things like sliding the keyboard on and responding to touch events were slower on Android. The default keyboard also seems much worse on Android, particularly the input of special characters and numbers. There are settings buttons built into the keyboard display for things like input options and predictive text, which just get in the way.
iPhone Home Screen
Specific to the Pulse, I also found I missed the proximity sensor built into the iPhone when making calls. Having to navigate phone “press 1 for sales…” menus with Android was horrible because it took several seconds to reactivate the display, get past the lock screen, show the keypad and then press a button. This had to be repeated when going down multiple menu levels.
But the biggest problem I can see is OS upgrades. The T-Mobile Pulse came with Android 1.5 and it is a relatively new phone. Relying on the manufacturer to release OS updates will be frustrating, particularly when they decide to stop supporting older devices. Apple does not have this problem because they own both hardware and software and whilst some features might be unavailable (e.g. compass, 3G, etc) you still get access to the core OS features regardless of the device you use.
That said, iPhone OS is more mature than Android and there are regular releases from Google. Google knows how to deal with developers and the openness of the platform works in its favour. It is far easier to start Android development than iPhone.
The integration of the Google Apps suite is a major advantage Android has. I much prefer using Android to interact with my Gmail, Calendar and Gtalk and have carried both devices for that reason.
Conclusions
Switching to Android is very compelling just for access to a more integrated Google experience. However, as a general phone for sysadmins, the iPhone is currently better. It has the full experience sorted – you can still use the Google Apps very well, plus all the other functionality needed by a sysadmin. Android completely failed for remote access, but that is more the fault of the app but it’s the apps that make the platform work. Google have done their bit for their own apps, now it’s time for 3rd party developers to get involved.
If you are making a buying decision now, my recommendation would be the iPhone. By the time your contract comes up for renewal in 18 months, the landscape will have progressed significantly and I suspect Android will be seriously competing for the attention of sysadmins by then, if not sooner.
I will soon be writing a followup post to this looking specifically at the apps available for sysadmins on both platforms.
Over the coming weekend we will be pushing out our statistics collection mechanism. This has been in the sd-agent source for some time but we have not been storing the data posted back. We will start storing this anonymised data as of this weekend (21st/22nd Nov).
The data that is reported back is agent version, machine type, processor type, OS, OS version and Python version. It is sent once each time you restart the agent.
We will be using this data to help our development (mainly choosing the right platforms to test on) but also to product generalised reports so we can publish some more interesting blog posts.
sd-agent v1.3.0 released in October introduced a switch to allow you to disable this. Enabled by default, just add the following line to your config to disable:
report_anon_stats: no
Let us know if you have any questions.
Last week our VAT (UK sales tax) registration came into effect. Unfortunately, the VAT rules were not drafted with international sales of software (as a service) in mind so we are faced with a relatively complex structure that must be applied to all our customers.
I spoke to our accountants to get an explanation of what we had to do and I present that below. Note that whilst this may help you understand what is necessary, it is tailored to our requirements and as such, should not be considered advice. Indeed, you should definitely speak to your own accountant to learn what you need to do. Tax is a very complex area and getting it wrong will result in fines.
I can recommend Complete Accounting Solutions as a company that knows the tech space very well.
The rules
The relevant legislation is VAT Act 1994, specifically paragraphs 7C(a) and 7C(b) of Schedule 5 which deal with “Electronically supplied services”.
This provides that VAT must be added to invoices where the customer is:
- based in the UK
- based in the EEC and is not a business
Further, there is a “use and enjoyment” override which means that whilst customers outside the UK/EEC would not normally be charged VAT, if they “use and enjoy” the service in the UK or EEC then VAT must be added i.e. they use our server monitoring service, Server Density, on servers hosted in the UK/EEC.
For the second point, we have to be satisfied that the customer is a business in the EEC by collecting their company and/or VAT registration details. And we simply ask customers if they a are “using and enjoying” within the UK/EEC for the exception.
Taken from the HMRC website, the 27 EU countries are the UK, France, Belgium, Luxembourg, The Netherlands, Germany, Italy, The Irish Republic, Denmark, Greece, Portugal, Spain, Sweden, Finland, Austria, Malta, Estonia, Latvia, Lithuania, Poland, Czech Republic, Slovakia, Hungary, Slovenia, Cyprus, Bulgaria and Romania.
just because you can do anything doesn’t mean you should do everything
- Andy Gadiel, Twitter
When you are selling to larger companies (and sometimes smaller ones too), a potential argument that might be raised either internally or with your sales pitch, is why they should buy your product instead of building it internally.
This post will look into the arguments from both sides and how you can address them. It has been written jointly by myself using the example of our server monitoring product, Server Density, and Oren Michels, co-founder & CEO of Mashery, an API infrastructure provider.
Before you even consider making the argument for buying your product, you need to ask yourself a few questions to tailor the pitch to your customer.
Q1: What does it look like from their standpoint?
You need to understand what your (potential) customer is going to get from your service. Are you selling them “great server monitoring” or “free revenue through an awesome addition to their existing product offering”? Are they interested in your product because it is something they will be using internally to manage their own infrastructure needs or something that will be given to their customers?
Once you know what they are getting from the arrangement, you will be able to properly understand what value you are adding.
Q2: Cater your licensing model to their situation
If you are selling for their own internal use then the licensing model will be different compared to if you are selling for them to resell your product, or even provide it integrated into their product offering.
The ROI question they are going to be asking is “when does building it pay off vs. buying it monthly”.
Some key arguments
We do this all day, every day. We are the experts.
The product you are selling is part of your core business. You will have spent time, effort and money developing it over a long period of time and have staff dedicated to continually working on and improving it. Your potential customer will be asking how easy it is for them implement, how long it would take to match your features and what resources would be needed to keep it maintained.
If it is something they need right now then there’s a clear case for buying because of the resources required to implement something in-house.
“In the same way you let the experts handle your e-mail servers (e.g. Microsoft Exchange) and phone systems (e.g. BT), we can handle your server monitoring.”
Everything is managed and supported by us
Providing a hosted service means the infrastructure is already in place to handle demand and future growth. Fixes to problems can be quickly deployed and the system is supported. The customer does not need to dedicate someone to ensuring everything remains operational as that is handled by you. This is one of the selling points of managed support services.
“Mashery will dynamically scale to support load spikes and long term growth so you don’t need to fire-fight sudden demand or pay for under-utilised capacity.”
You should be working on your core business
Choosing to develop something already available now has opportunity cost in terms of what else could be worked on instead. It takes time to get a team up to speed on something new and the time-to-market will be much greater than if an existing solution is brought in right now.
Costs
What is the true cost of building in-house? Internal resources are not free.
Taken from the Mashery website:
Conclusion
Selling to larger companies is always difficult because they have a lot of decisions to make before committing to one product. Whilst the above can serve as a guide, the important thing is to prepare yourself for each situation so you can have the right things to say at your fingertips when you need them.
