Entries tagged with "web"
Development of Personal Site: Part 4 (Django)
Though at my work I am java developer I am fond of python, and wanted to use python for developing dynamic parts of the site (like blog, poll, flickrbrowser).
At first I decided to find all applications that I need and combine them into single web site with consistent look and feel using paste . To my surprise most applications that I have found were just CGI-based web applications or scripts and I decided that it quite a lot of efforts to consistently combine them together.
I continued my investigation and found PyBlosxom . While this blog engine is quite simple and extensible by plugins it is not easy task to configure it properly. Besides at least four blogs mentioned on its users page switched to WordPress, that is written in php. But I don’t want php. One more important point that resources of my VPS are quite limited and it is not very wise to run php+mysql only for blog and python for other applications that I am going to develop.
Still investigating I found post in GvR’s blog about python web frameworks and somebody mentioned django in comments to this post. I had heard about it before, but thought that there is nothing special with it, just some combination of ORM and template library. But after reading very good opinions about it I decided to give it a try.
I was really amazed by this great framework. This exactly what I need. It allows to combine several applications in single site with little or no effort. Though all applications that you are able to see on this site were written by me almost from scratch (and this is different from my original intention to reuse applications as much as possible) it was so easy to create them and allows me to learn django better.
There are a lot of speculation that python should be promoted more aggressively and I think that django really can help python to become more widely-used language.
I imagine that as it becomes more popular a lot of django applications (like blog, forum, poll, CMS, image gallery, etc) will appear and instead of writing everything from scratch it will be possible to specify list of necessary applications (as eggs), configure them, write some base templates and CSS styles to unify look and feel and receive great dynamic web site.
Posted by ksh on April 14, 2006 | 5 comments | development, django, python, web
Development of Personal Site: Part 3 (Software for server)
If you read my previous posts about development of personal site you probably already guessed that I have installed gentoo linux on my VPS.
Gentoo is my favorite linux distribution over there. I am using it already for 3 years at home, for almost 1 year at work and for 2 months at VPS server :) and not going to change it. Before I have found gentoo I tried RedHat, now I am administrating Debian and RHEL servers at my work, but I’ve never feel enough comfortable with these distributions. I am not sure what are the real reasons for it, probably this is because of lack of knowledge and practical experience with particular distribution or may be I just do not like them and has mental barrier that prevents me to use them efficiently. But anyway I have chosen gentoo and want to describe some details about what software and why I have installed.
Because VPS memory (64Mb RAM + 128Mb swap =198Mb total), CPU and hard drive (3Gb) resources are quite limited at first I doubted that they are enough for source-based linux distribution where you need to compile every single package from sources and also keep a lot of development files (in /usr/include) that are not usually necessary for binary distributions. But it turned out that it is more than enough. Upgrading all packages (including glib and gcc) to the most recent versions and installing all necessary software is a matter of several hours. Additionally I’ve performed some configuration and was able to decrease disk usage. Here are some tips:
- add nodoc, noman, noinfo to your FEATURES in /etc/make.conf (man make.conf for more details)
- make sure that you have configured RSYNC_EXCLUDEFROM (Diverting from the Official Tree). You certainly don’t need ebuilds in such categories like x11-base or kde-base on your server and you can safely remove all such ebuilds from your PORTDIR
- do not allow make to execute several commands simultaneously: MAKEOPTS=”-j1”
While portage tool itself is very convenient it still lacks some helpful functionality. Fortunately there are some tools that fill this gap: gentoolkit and flagedit. I would recommend every gentoo user to install these packages and learn how to use them.
Every site need HTTP server. I have chosen apache, mainly because I want to be able to access my subversion repositories via HTTPS. Another good HTTP server is lighttpd .
Though mail server is not really necessary it is very convenient to have it. I have installed postfix. It is quite simple to configure (it is very good tutorial in gentoo wiki) and has all necessary features that I need out of the box. BTW if you have VPS you can have very cool email address, like yourname@step-inside.org. but I decided that my mailbox at gmail is enough for me and also I don’t want to fight with spam, so postfix is used only for local mail.
Everybody likes statistics. So do I. I have chosen awstats package. I recommend to install Geo-IP package and enable geoip plugin for awstats: you will see what countries your visitors come from. It was necessary to tune apache configuration slightly so awstats can understand its logs: CustomLog logs/access_log combined.
You most probably want to install logrotate , if you don’t want to find out that log files have eaten all your free disk space.
It is very important (and sometimes interesting) to know what your server is doing all those long and boring days and nights when it doesn’t server content of your site. I recommend to use logwatch for this purpose. It will check your log files every day and send summary report to your mail with description what happened. It requires some tuning after installation, particularly I have changed log directory for apache (in /etc/log.d/conf/logfiles/http.conf), and specified that cron and postfix do not use log files and instead rely on syslog service, so logs of these tools should be extracted from /var/log/messages file (in /etc/log.d/conf/services/cron.conf and /etc/log.d/conf/services/postfix.conf). After these changes still some tuning is necessary, but I haven’t figured out what exactly. Probably I will blog about it in future entries.
I was amazed that there were so much breaking attempts: dictionary attacks by SSH happen every day, when I installed apache I immediately started to receive a lot of requests for such URLs like:
/articles/mambo/index2.php?_REQUEST[option ... cho%20YYY;echo| /blog/xmlrpc.php /blogs/xmlsrv/xmlrpc.php /cvs/mambo/index2.php?_REQUEST[option]=com ... cho%20YYY;echo| /drupal/xmlrpc.php /index.php?option=com_content&do_pdf=1&id= ... cho%20YYY;echo| /index2.php?option=com_content&do_pdf=1&id ... cho%20YYY;echo| /mambo/index2.php?_REQUEST[option]=com_con ... cho%20YYY;echo| /phpgroupware/xmlrpc.php /xmlrpc.php
This brings another important issue: security. Gentoo developers pays a lot attention to it. So I have scheduled emerge sync and receive email with warning if my system is affected by any GLSA . Also I have disabled root logins via SSH (Tip: do not forget to add your main user to wheel group before doing it).
Other software packages that I have installed (and you can also find them useful):
Posted by ksh on March 30, 2006 | 0 comments | development, gentoo, linux, web
Development of Personal Site: Part 2 (Domain Name)
In this post I am going to describe process of choosing domain name, registering it and making it to be resolved to static IP address of my VPS server. I ran into some problems when was doing these tasks and want to describe these issues along with their solutions.
No one in the world really “owns” a domain name except the Network Information Centre, or domain name registry. But if you want to have one, you should register it. There are a lot of companies that offer domain name registration (so called Domain Name Registrars) for affordable price (just search for “register domain name” in google). UnixShell (my VPS provider) doesn’t provide such service, they decided to go true Unix way: do one thing and do it well.
Real problem with registering domain names is that the name you want to have can be already owned by someone. Alas, this was the case with name for our site. Originally we want to call it stepinside.org, but ended with step-inside.org (I also thought about calling it sepinsi.de :).
One thing that turned out to be harder then I expected was to associate my domain name with static IP address of my VPS server, but now I know much more details how DNS works :) And I want to share my knowledge with you.
Originally I though that I would be able to login to the site of my registrar and using web interface configure my domain name to be resolved to my static IP address (i. e. create A-record in terms of DNS). Unfortunately they don’t provide such functionality or I am so dump, that haven’t figured out how to do it using provided web interface (but this means that average user also unable to do it).
Fortunately this task can be solved without help of your domain name registrar. Every domain name has so called WHOIS record. It contains such important information as owner of domain name, details about registrar, registration and expiry date and so on. Your registrar is responsible to maintain this record and you should be able to edit at least some details of WHOIS record of your domain name using web interface.
The most important thing for our task is a list of nameservers that are responsible for resolving domain name. You need to specify list of nameservers that are able to say that your domain name is resolved to certain IP address. For more details you can check very good overview of How DNS works .
You have two possibilities for such nameservers:
- run your own nameserver (such as bind or djbdns)
- use one of free DNS services, like EveryDNS.net
I’ve chosen the second one, because it is much easier :)
Posted by ksh on March 24, 2006 | 2 comments | development, web
Development of Personal Site: Part 1
Nowadays it seems that only lazy doesn't have personal page. So some time ago I and my brother decided to create personal site. I as being programmer of course wanted to create it by myself.
This entry opens up series of blog posts that describe various technical details about way how I have developed site you are browsing now. When I first thought that I should have my own web site I had general knowledge how web is working, and general idea how this site should look like, but I hadn't know enough those low-level details that are necessary to create something by yourself. I hope that my description help somebody who is in the same situation.
I knew that this site should fulfill following requirements:
- It should be dynamic. So if I need some sort of online application that helps me in my work I should be able to develop it by myself and deploy to the server.
- I want this site to be written in python, because this is my favorite programming language.
- I didn't want to develop it from scratch, instead it should be based on already existing framework or set of applications, because I haven't enough time to write all functionality by myself (even it is so easy to create frameworks in python) and I am quite lazy and want to use others people work instead doing something by myself :)
Server (hosting)
The first thing that is necessary for every site is server where it lives. Though there are some providers that offer python-hosting I've decided to maintain server by myself. At at the beginning this require more effort, but having your own server in the Internet has a lot of advantages:
- You can install any software you want and configure it for maximum flexibility.
- If there are some problems you can fix them immediately by yourself, instead waiting for provider's help. Though there are many people who would think about this fact as disadvantage :).
The best choice in such a situation is VPS: you don't need to think about hardware, but you have full control over software. VPS provider of my choice is UnixShell. I've chosen it, because
- They offer great service for acceptable price.
- Their site is simple (but changes very often :), free of various buzzwords and annoying ads and contains all necessary technical information that are interesting for people who know what they need.
- This was the only provider I've found that allows me to install gentoo linux as operating system of my VPS (more on this in the next posts).
- I like their name :)
That's all for now. In next entries I am going to describe details about registering domain name, isnstalling server software and choosing right python tools to develop this site.
Posted by ksh on March 13, 2006 | 1 comment | development, web