CrashPlan announced a couple days ago that they will no longer offer their services to consumers. If you were using their service, you have about 14 months to find an alternative.
I have already started – I uninstalled CrashPlan and looking at alternatives.
What are your options now? You may want to look at BackBlaze, they have a nice looking service.
Rspamd is supposed to be a high performance spam filtering solution. Of any project I have worked with, it has the worst documentation of anything I have used. Obviously this is the reason the product has not been widely adopted. You may have the best software in the world, but if no one can figure out how to use it…
View Stats
rspamc stat
Command will output some details, including how many messages scanned with a breakdown of classifications. Also lists details of fuzzy hashes and bayes information.
View Configuration
rspamadm configdump
Command will dump out the active configuration of rspamd. Very useful since rspamd uses a general configuration file, then has local configuration files that merge and also override configuration that replaces default configuration.
You can drill down and look at individual specific configuration element (there are many), here are a few examples.
rspamadm configdump logging rspamadm configdump regexp rspamadm configdump classifier
The following links are places I have found with good general information or documentation on RSPAMD.
I have previously written two posts about the cost of using Amazon AWS, one way back in May 2010 and one in August 2015. In both cases the costs involved with running a rack full of servers 24/7 365 was much more expensive than we could do it ourselves hosting equipment in a data center.
A project I’m working on is growing and we have a couple options that we are considering. Our volume increases a lot during USA working hours, specifically Monday – Friday 8am – 1pm. We are considering the following two options:
1) Provision Amazon EC2 servers during peak times, so 5 hours a day and 5 days a week.
2) Purchase a bunch of Intel NUC DC53427HYE computers. These is not server grade hardware, but they do have vPro technology which allows them to be remotely managed.
We have purchased 1 of these little Intel NUC machines for testing, purchased new in the box for $180 – plus shipping so about $200 for the base unit. We need 4gb of RAM and 120gb of disk space.
Configured the NUC is costing about $290 ($200 NUC + $60 SSD + $30 RAM). 10U at our data center with 20 amps of power and 30Mbps bandwidth. We should easily be able to power 30 of these little NUC machines with 20 amps of power. The data center space will cost $275 per month, for 24/7 operation with 30Mbps of bandwidth based on 95th percentile usage.
Lets run some CPU benchmarks on these little guys and see how much Amazon cloud hosting would cost us for the same compute power running only 5×5. It should be a lot cheaper to go the Amazon route… but lets find out.
The pricing / servers have not changed much since I ran the benchmarks last year. These benchmark were run in Amazon US West (Oregon) using Performance Test 9.0, which is a different version than last year so the scores are a bit different.
Instance Type:
c4.large – 8 ECU – 2 vCPU – 3.75gb RAM – $0.193 per Hour
Windows Server 2016
CPU Mark: 2,925
Disk Mark: 791 (EBS Storage)
Intel NUC DC53427JUE – Intel i5-3427u – 4gb RAM
Windows 10 Pro
CPU Mark: 3,888
Disk Mark: 3,913 (Samsung 840 EVO 120gb mSATA SSD)
Wow, the benchmark performance for the Amazon instance is quite a bit lower than the basic Intel NUC.
The total CPU score of 30 of these Intel NUC servers would would be 116,640. To get the same computer power out of the c4.large instance at Amazon we would need to boot up 40 instances.
Lets run the costs at Amazon. We would need 40 servers at .193 cents per hour. We need them for 5 hours a day, 20 days a month. So the math looks like this.
40 servers x 0.193 = $7.72 an hour
$7.72 x 5 hours = $38.60 per day
$38.60 x 20 days = $772 per month
In addition we need to take into consideration the load balancer, bandwidth and storage. We are using an estimate of 3,300 GB per month inbound and outbound to get our estimated pricing, this is only a fraction of the total bandwidth we could theoretically move on our 30Mbps line from the data center.
Load Balancer Pricing
$0.025 per hour = 5 hours x 20 days = 100 hours = $2.50 per month
$0.008 per GB = $0.008 x 3,300 = $26.40 per month
Bandwidth Pricing
3,300 gigs outbound bandwidth = $297 per month
Storage
Sorry Amazon, your pricing is so complex I can’t even figure out how much the disks are going to cost to provision 120gigs per machine for the 5 hours x 5 days operation…
So a rough cost is going to be somewhere in the range of $1,100 per month for only 5 hours a day, 5 days a week.
Lets look at a 1 year investment.
Amazon = $13,200 per year
Buying 30 Intel NUC servers and hosting them.
Intel NUC x30 = $8,700
Hosting = $275 x 12 months = $3,300
Total to buy and host 1 year = $12,000 for 1st year
Paying for the hardware up front, running the servers 24/7 still comes out cheaper than going to the cloud with usage of only 5 hours a day and 5 days a week.
Lets lengthen the time to 2 years.
Amazon = $26,400
Own & Host = $15,300
And the 3 year view?
Amazon = $39,600
Own & Host = $18,600
Incredible, I thought going to the cloud would be more cost effective than buying and hosting our own equipment when we only need the extra capacity for a limit amount of time per day.
It was back in 2011 when Supermicro released the 2015TA-HTRF.
It is a 2U rack mount fat twin. It contains hot swap modules with two Atom-based computers. In total, you get 8 Atom computers and 24 2.5 inch hot-swap hard drive bays.
I want to build a Ceph cluster and of all the hardware I have seen this looks like it would be the most cost effective. Since I love buying used hardware this is what I would like to get.
The problem is, it must not have been a big seller. I have been unable to find even a single used unit for sale… I’ll keep watching eBay but do not have very high expectations in finding these 🙁
Check out the 2U – 4 node servers the Supermicro SuperServer 2026TT-H6RF. Nice looking upgrade from the servers I have been using and they are very cheap (like $200) on eBay as of June 2016.
This is mostly a note to self, and anyone else who builds racks based on off-lease servers.
Learned about these when I found some link to them while reading ServeTheHome.
I’m working on setting up a bunch of servers, so I used the Windows System Image Manager to create an image with answers to all the questions of a basic install.
I used VMWare to boot the ISO image I was creating to test.
Placing the autounattended.xml on the root of the DVD ISO gets the image in VMWare fully setup.
Try it out on the actual server and nothing works. I am prompted just like a normal install. It is not reading the autounattended.xml file at all.
Turns out the autounattended.xml file will only work if the BIOS considers the device removable, like a USB thumb drive or something. The BIOS I am using is mounting the ISO as a fixed disk so the Windows installer will not use the autounatteended.xml that lives in the root of the ISO.
Good news, there is a solution to this problem of the Autounattended.xml not being picked up by the installer. The process inegrates the Autounattended.xml into the boot.wim file so it is always picked up.
You need the following items beforehand:
Then do these steps:
Mount the boot.wim image located on your USB HDD disk (in this example H: is the USB disk) using ImageX from WAIK
imagex /mountrw H:\Sources\boot.wim 2 C:\temp
(assumes you have a folder “C:\Temp”)
(the number 2 stands for Index 2 within the boot.wim image)
Fire up your Windows Explorer and navigate to C:\Temp. You will see your boot.wim image mounted and all. Drop your “autounattend.xml” file you created directly into this folder (right next the the Setup.exe file)
Close Windows Explorer and unmount the image:
imagex /unmount /commit c:\temp
Amazon gave me $100 gift card for AWS services a couple months back. As much as I love the concept of cloud the numbers have never worked out for me. My gift card is going to expire soon, so lets burn up some credits by bench-marking some of their AWS EC2 instances.
I’m most interested in CPU and disk performance of various instances since servers do not need high end graphics. All tests were performed on Windows 2012 R2 Standard Edition (64 bit) using PerformanceTest 8.0 software.
The Amazon pricing per hour is for the N. Virginia area, which I believe is cheapest zone they offer.
Instance Type:
c4.large – 8 ECU – 2 vCPU – 3.75gb RAM – $0.193 per Hour
CPU Mark: 2,708
Disk Mark: 787 (EBS Storage)
c4.2xlarge – 31 ECU – 8 vCPU – 15gb RAM – $0.773 per Hour
CPU Mark: 9,485
Disk Mark: 1,017 (EBS Storage)
c4.4xlarge – 30 ECU – 16 vCPU – 30gb RAM – $1.546 per Hour
CPU Mark: 15,680
Disk Mark: 998 (EBS Storage)
c3.xlarge – 14 ECU – 4 vCPU – 7.5gb RAM – 0.376 per Hour
CPU Mark: oops… have to redo this one
Disk Mark: 910 (2 x 40 SSD)
Non Amazon Systems:
Supermicro – XEON L5420x2 – 16gb RAM
CPU Mark: 4,445
Disk Mark: 1,385 (Samsung 840 EVO 120gb)
Crunching The Numbers
I want to compare prices of running in the cloud to the Dual Xeon L5420 processors, which are available for very cheap on eBay. Perfectly good used servers, slap some new SSD into them stick them in a datacenter and run them until they die.
The closest match offered by Amazon is the c4.2xlarge class machine, which has a CPU mark of 9,485 vs the dual Xeon’s score of 6,734.
The cost to run in the Amazon cloud would cost you $556.56 per month. That is just the machine, it does not include extras such as a load balancer, VPN or bandwidth.
The cost to run a 1/4 rack (10 machines) would be $5,556.60 per month. If you need and entire rack it would cost you $23,375.50 per month.
You can get your cost down quite a bit if you are willing to commit to a long term agreement of 1, 2 or 3 years with Amazon. Once you commit to a specific instance you can’t change, so calculate your usage requirements before committing.
I started using memcache to store sessions, rather than having PHP store them on disk. The server hard drives are SSD so I never noticed any performance issue sorting them on disk, but I did not like all those files filling up my /tmp space.
Once moved to memcache, then you have the issue of redundancy. If you have more than one server handling your traffic load, you need something to maintain a sticky session or the user would be logged out of your site (or session information lost) if they move between servers.
Doing some reading there seems to be a lot of bad information out there about exactly how to setup session redundancy across multiple memcache servers.
On a lot of sites I found this syntax to use:
tcp://127.0.0.1:11211?persistent=1&weight=1&timeout=1&retry_interval=15
I think that syntax is not correct, but it is that way on many sites. According to the documentation you would not encode the &. In addition all the values they are listing are the defaults. So odds are good those params are not valid like that, but it still works because the values are the default.
One of the better articles I found online was this one at DigitalOcean.
How To Share PHP Sessions on Multiple Memcached Servers on Ubuntu 14.04
His configuration is a bit different than mine, since the OS is different. A couple things to emphasize if you are trying to set this up.
In his example using three servers he says to place the following on each server:
session.save_path = 'tcp://10.1.1.1:11211,tcp://10.2.2.2:11211,tcp://10.3.3.3:11211'
The order here is important, I think a lot of people will want to change the order of the servers, placing the local server first. Don’t do that! In order for the redundancy to work correctly the session.save_path must be the same on all servers. Do not worry about the order, as PHP must contact each server to write the session data anyway.
Seems linux people don’t like RAR very much, it is not easy to install from most package managers it seems. Some used to have it, then removed it because it was not free.
Then some people come up with limited free versions etc. etc. I ended up just getting it direct from the source @ rarlab.
Move to your tmp folder and download from rarlab site.
cd /tmp wget https://www.rarlab.com/rar/rarlinux-x64-5.8.b1.tar.gz
Extract the downloaded file (extracts to rar folder).
tar -zxvf rarlinux-x64-5.8.b1.tar.gz
Move the files to your /bin folder so you can execute rar and unrar from anywhere on your system.
cp rar/rar rar/unrar /bin
That’s about it. Now you can actually use the full power of rar on your linux box.
I love the concept of cloud hosting, but the numbers just don’t work. A few years ago I did a calculation of CPU power vs. Amazon compute units.
The bottom line was if you needed a server online 24/7 it was just not cost effective. I know Amazon has a lot of clients and people can make it work – but I can’t get the numbers to work.
I tried again today to look at some numbers. I’ve got a pair of email servers and in the last 24 hours consumed about 74gigs of bandwidth. Amazon now has free inbound bandwidth so lets cost only outbound bandwidth at 35 gigs per day, or 1050gigs per month.
Just for bandwidth alone Amazon would charge $124.20 a month.
That does not include the TWO servers that I would need to pay for. The servers are SuperMicro i5 based machines, not exactly sure where the performance would line up with Amazon cloud instances but I will estimate conservatively at c3.large class servers which have 3.7gb ram and 7 ECU units. Lets check the numbers under a long term contract with 24/7 usage.
The two Windows based servers would cost $270.68 per month.
How about adding DNS to the list. I am including a snapshot from one of my DNS servers (I have 4 total), you can see it has been online for 32 days and has processed 115,937,886 requests or 3,623,058 requests per day.
The DNS service (Route 53) would cost $58.50 per month.
So lets recap:
Servers: 270.68
Bandwidth: 124.20
DNS: 58.50 (I actually have 4 of them, not just one)
For a grand total for the month of $453.38.
Hate to say it Amazon, but I pay about $100 to host my two servers every month. The DNS software was a one time cost, years ago. The servers are SSD RAID based with 8gb of ram, total cost was about $1000 each and should last at least 3 years but probably much longer ($27.75 a month if they last 3 years).
So the numbers for a cloud based service still do not work for me 🙁