As I was doing the most popular website on the election

Friends, I want to tell you the history of the project president2012.ru. The website is a leader on the subject of presidential elections in Russia for 3 months it was visited by nearly 2 million people in its peak had up to 20 thousand visitors online and more than 1 million hits a day, were 3 servers in the cloud.

Interesting moments in the process of working on the project had accumulated so much that to keep this knowledge I can't, and I want to share with you. I understand that much of what I'm about to tell has already been described qualitatively in specialized sections, and nothing new I will not open. My goal rather is to separate the nuances to convey the atmosphere that prevailed in the work on the project. And for the millionth time with success story to show to people like me people that everything is possible and everything is in their hands.

To whom it may be interesting? All who in any way engaged in the creation of thematic websites and hopes to work with high load projects.

I hope the political hype verse enough so that readers could appreciate my article without the influence of their political affiliation.

Will lead the story in chronological order.

the

Prehistory

It all started in February 2010 when on our local forum (I live in Kemerovo) ran the news that the former owner did not renew domain president2012.ru and it is put up for auction. Without hesitation, I took part in the auction and, lo and behold, was the only buyer with the final price of 100 rubles!

Until April 2011 the domain was plug in a list of potential candidates, I only base pushing on "elections 2012". In March 2011, I commissioned the development of its programmers and designers (I'm the Director of WebStudio and another couple of IT-shnyh companies) and 6 April, the website was released "into the wild". The main functional pages of candidates with their description, programs, etc., as well as the ranking of these candidates by voting of visitors. Secondary content aggregated by subject news, articles and videos (with the obligatory reference to the source and observance of the requirements of this source to copying of materials). Was attracted by the content Manager to aggregate content and moderating user comments.

the

warm up

The reaction of Internet users has not kept itself waiting, every month the number of visitors doubled, and by August 2011 the total number of comments has reached 2000, and faced the problem of spam/flood/flame. Until August commenting was anonymous with captcha. To lose such a wonderful source UGC did not want, so for maximum ease of commenting for visitors, I chose Loginza, which allows to simplify the work with the authorization API of the social networks and different services. I will not describe here the complexity of its integration in the website, but even with a minimum programming knowledge is a matter of a few hours.

On the same day, the number of comments has decreased several times, especially because the comment was not anonymous and their words have had to theoretically carry more responsibility. But after a few days, visitors got a taste of innovation and the volume of commenting back. For me, simplified the task of moderation, and the quality of reviews has improved meanwhile. Of course, the closer the date came the elections, the more the commentators went beyond all conceivable limits and had to make more flexible system of moderation, mass deletion of posts, mass ban clones, etc.

the

Vote

It deserves a separate description of the voting process. The supporters of the candidates used all possible ways of cheating votes: lookup referrers, useragents, etc., they sat for days on the discernment of captcha, used a proxy, even TOR is used. In the end, the only netregistry way to somehow limit their zeal was selected for confirmation of the vote via SMS. Because to a large mass of mobile numbers is much more difficult, than to change IP or register a new account in social networks.
The choice was two options:
the

the
• After clicking the button "Vote", to offer to enter your phone number and send an SMS with a confirmation code, which in turn is entered in the adjacent field. Minus of this variant — while maintaining the speed of voting by visitors (a few thousand votes in a day), I would simply not have coped with the budget load, because for each outgoing SMS would have to pay. But, nevertheless, the market of service delivery studied and found two of the cheapest and most convenient option: smspilot.ru and sms24x7.ru. Both allow you to send SMS not more than 30 cents apiece and with an arbitrary sender name, for example with the name of your brand.
the
• the Second option is to issue a unique voting code which you want to send on short SMS-number for verification of their vote. The main disadvantage of this method — the payment for sent SMS that discourages many voters. On the other hand, this method does not reduce your budgets, but on the contrary, allows to compensate a part of expenses. I tried to minimize the cost of voting the cheapest short number from the SMS cost about 3 rubles. Plus held "explanatory" work on the voting page, describing the reasons for the choice of this method, plus a promised part of the proceeds to send to charity. Now, by the way, choose an orphanage in the city for the fulfilment of that promise.

Here I was waiting for the next rake. Were originally selected SMS service A1Pay, but at some point they, apparently, something terrified, covering my service, the "warning" about this ticket in your interface Friday night(!), and no notifications for email etc. the Message was this:

Please urgently remove our numbers from the website president2012.ru. If during the day the rooms are not taken, we will disable the root prefix and Your account.

I found this ticket on Sunday evening when the count of the vote for a long time did not change, but hundreds of people sent in their SMS "nowhere" to the blocked service. In "the clock" technical support and could not get through for hours. A long correspondence with the "security service" A1Pay managed to achieve only this explanation: "We are an apolitical company and do not work with such resources."

I had the selection of a new partner. In the process of this selection I came across the fact that most mobile payments represent "sharazhki" with a disgusting support and absence of any guarantees. So, for example, advertised on searchengines Smsbill.ru gave me a "Service is not authenticated, information: icq xxxxxx". Turning to a specified ICQ, I got the answer in style "And you have obtained the consent of the presidents?" and "We don't work with such grey on sites like yours." The person from "technical support" automatically enrolled me in "unworthy". Oh well, still found the operator, who, after a round of negotiations agreed to start my service smsdostup.ru. After the voting problems did not arise.

I must say that with the introduction of SMS confirmation, the number of votes fell several times, but the quality of them has grown considerably. The quality is evident in temporal, geographical, license distribution. It is seen that the voting began to occur more evenly without bending and twisting.

the

Success and its fruits

And the growth meanwhile continued. Any political event led to bursts of traffic. Links to the site in social networks periodically led to the viral wave. A striking example — the wave which commenced after the publication of the me program of the candidate Peunova Svetlana Mihailovna. The day my site hit the tops dirty.ru, vott.ru, pikabu.ru, onliner.by, nigma.ru, leprosorium.ru blogs teh-nomad exler.ru got +400 links from Twitter, hundreds of links from FB, VK, OK. And even to me quoted.
By the way, SMO-work on the site resulted in a great dissemination of information throughout the network. The greatest effect was obtained, of course, from Facebook, but other social media did not disappoint. Especially liked the Twitter, where every day there is a to hundreds of messages urging them to vote. The most promising discussion my site I tried to maintain, gathered feedback, worked with claims, etc.


Starting around December 2011, the website president2012.ru ranked TOP1 in almost all queries relating to the election, both in Yandex and in Google. By the way, about competitors in the last elections of 2008, the top site was vibori.net, which I this time left far behind, giving them only a fifth of the possible traffic. The other "competitors", to my surprise, not even trying to be first, not promoted, are not upgraded their sites for emerging realities. So, many of them have not even updated the date of the upcoming election, when it was moved from 11 to 4 March 2012.

Constantly growing traffic led me to the fact that in December 2011 the maximum rate at my hoster nic.ru ceased to satisfy the requirements of the site of memory and individual memory settings, the host does not provide. Began the periodic drops. I set up ngnix configured to return static content. This reduced the load in 2 times, but after a few days already and this is not enough. Site is moved to the cloud clodo.ru, allocating 2 GB of RAM and 4x2267MHz processor. Until March 2012 it was enough to hold on.

the

New challenges

On 23 February I received another bandwagon. Yandex has launched a website vybory2012.yandex.ru, putting it first for all search queries containing the word "elections" and thus violating your own rules, which translates to Yandex webmasters. By this action, they took about 15% of the traffic — share is small, but put up with it is not wanted. Wrote to them the message in blog what a day got hit even harder under the pretext of correcting the error. This time they removed your site from the issuance of "broad" query "Elections", but put it in the results at a much more frequency of the many queries with the words "elections", "candidates" etc. to Surrender, I wasn't going, knowing that it's probably just the initiative of individual members of the team of Yandex. I wrote a complaint to the Yandex accounts on Facebook, LiveJournal, Twitter, wrote a letter in support of the search. And after a couple of days justice has triumphed. Yandex has left your website the first only in a very narrow queries, and in other — "mixed" results on napervi positions. By the way, after that I got more respect Yandex.

27 Feb to me suddenly opened to the fact that according to 19-mu to the Federal law on elections of the President 5 days before the vote is prohibited to publish the results of a public opinion poll. I was faced with a dilemma — or to break the law, because in fact, my vote could be equated to the survey, or to close a substantial part of the site in those days, when in fact this information people massively and will be looking for on my website. And I turned! Stopped the update, promising the voting results update on March 5, and everywhere on the site wrote that the date of publication of the ranking of candidates on February 27. This is fully consistent with the requirements of the law, and the site could continue to work in peace.

the

In ecstasy

So, come the official day of voting. By 3 March traffic rose to 80tys people a day. I began to predict what will be the load on 4 March. Graph site traffic vibori.net in 2008, by extrapolation, gave me a forecast of 200 thousand. visitors on the day of voting. Then I did not know that this is a weak forecast and all will be even more interesting.
On March 3 I am most zakeshirovalo, sredstava CMS (MODx) all pages of the website (by the way, browser caching, I had already configured long ago), increased the RAM to 3Gb, and rendered static content on a separate server, and quietly went to sleep. Waking up at 11 am (MSK+3) I found that the servers are already at the limit and need something to do. Fortunately, the site was on a cloud and I had opportunities for maneuver.

The first thing allocated the database on a separate server. Helped, but not for long.

Increased the RAM to 8Gb and the number of CPU cores to 8. Another hour goes by was won.

And then woke up in Moscow! It was scary, the number of online visitors has reached 6000 and is constantly growing. It was the turn of more fine-tuning, ngnix. Never did, but Habr saved good manual. Increased the number of worker processes to 8 on the number of cores, the number of connections up to 8ths, limited connection duration to 15 seconds. In sum, changing the settings ngnix gave an increase in speed of return of pages in 2 times.

It was like a game of tag. Each new batch of changes to the server given the improved performance and automatically brought a new wave of visitors, again reducing performance.

My experiences on Twitter

Next, began tightening caching. The most visited page with rarely changing content saved as html files, completely removing the php part, and has prescribed the results in .htaccess. Site inside more and more distant from the starting of the engine, and turned to samostalnog and cached frankestein. But such was the price of the high-speed decision-making and speed of their implementation.

Parallel to increased iron power. By the end of the day I worked on 3 servers on two 8-gigagertsevyj cores and 8Gb of RAM: web filserver the database server.

Then I found a big long queue of executable database queries. Started to optimize the SQL for pages that have not been so harsh cached. Some queries, such as counting the number of reads, are generally removed, so as not to waste time on optimization is not the most essential functionality. Generally, in those moments when every second counts, the brain understands what is important and what can be ignored. At the same time, podumyval MySQL, reduced wait_timeout to 10 seconds, increased the max_connections to 1000. Claims to developers I had, because before them was not the task of vysokodispersnom website.


The beginning of each hour accompanied by a sharp growth, related, apparently, that was opened or closed polling stations in another time zone and people were going to look at the statistics (although output quickly according to the law, I could only percent turnout). And now, at a time when the closed areas in the main time zone of Russia — Moscow — the server broke down and went to bed. At the time the site was online about 20 thousand. people. Call tech support Clodo, understanding and apologetically say that the server died a heroic death, and they to me and say: "we Know it's our fault (!), within 10 minutes to fix." Did not argue with them, and 40 minutes later the website came back to life. Not much worried, because knew that, and so has fulfilled and exceeded all plans, and expect that the site will fall much earlier and for longer period.

Since the restoration of the activity of visitors began to decline and I could easily update the news section and videos, and after the vote in Kaliningrad uncorked a bottle of red wine and together with his wife had celebrated in his "campaign headquarters" end of a perfect day. The feeling was that the President chose me. Blood was running a mixture of adrenaline, dopamine and endorphins.

the

Results

In the following days, attendance slept much sharper than grew, and the holiday stopped at around 7-8 thousand visitors a day.

Final results:

During the period of active vote (3 months) the website was visited by about 2 million. people
Of these, about 500 thousand — in days from 8:00 March 4 at 8:00 on March 5 (UTC).
At the same day has been viewed about 1 200 000 pages.
At the same day outgoing traffic: 0.5 TB.
Spent on development and support around 60 000. Of them: 33 000 RUB. on the development of, 8, 000. hosting, 19 000 RUB for content and other support.
Received from advertising Google Adsense is about 190 000.
Experience: invaluable.

Summary

The conclusions that I made from this project:
the

the
• Event-driven projects can bring a good profit in spite of their brevity in time, resovist.
the
• it is Important to work actively with social networks, how to encourage UGC, as well as to increase the impact of PR and enhance your SEO.
the
• it is Important to track all of their competitors and always one step ahead of them offer convenient and useful tools to users.
the
• it is Important to focus on certain parts of your project and adjust an entire advertising campaign. So, my concentration was in the rating and commenting of candidates. I did not create characteristicly policyportal.
the
• If you predict a rise interest in your project — prepare for this growth, not to be in a situation when the demand for your service/product is, and stupid you can't satisfy due to lack of resources.
the
• and most importantly — you need to have a clear structured plan of action for the project, covering all aspects of its existence and development, and then remain to believe in yourself and everything!

PS looking forward to your questions, reviews, criticism on every aspect of the project — from idea to technical implementation.

P. P. S. the Administration please do not consider this post as political, if the limit on the policy still remains. I myself am apolitical person and have tried in the text to avoid all sharp corners.

Article based on information from habrahabr.ru