Source control (version control) in Ubuntu
Apache + SVN (subversion) + web +
From step 1 to 5, we will set up Apache and enable SSL service that provide a secure access to the repository via Web interface (https). Also, other protocols we can use are svn:///, file:/// to get the access to your repository.
1. Install Apaches (LAMP):
Apache is the most commonly used Web Server, at least in Linux system. To install it, hits
and choose LAMP server
To manage php, you should install also phpmyadmin (choose Apache2 when install phpMyAdmin)
sudo apt-get install php5 libapache2-mod-php5 libapache2-mod-auth-mysql php5-mysql phpmyadmin
and install subversion-tools, libapache2-svn
- To make sure Apache2 is properly installed, check: http://localhost/
- In Apache2, the config file is nolonger httpd.conf, but /etc/apache2/apache2.conf
2. Enable SSL:
We use a2enmod command to enable an Apache2 module (to disable, we use a2dismod). Here, we enable two essential modules: mod_rewrite and mod_ssl
sudo a2enmod rewrite
sudo a2enmod ssl
NOTE: If you can’t enable mod_rewrite, edit the file /etc/apache2/sites-available/default, and change AllowOverride None to AllowOverride All.
sudo /etc/init.d/apache2 restart
#sudo sh -c "echo 'Listen 443' >> /etc/apache2/ports.conf"
- Port 443 is generally reserved for SSL
3. Generate Certificate:
A secure connection requires the two ends (server-side & client-side) to have some identification mechanism.The client can be anonymous to the server, yet the server need to be certificated.
One way to generate a self-signed certificate key on the server-side is to use make-ssl-cert program:
sudo apt-get install ssl-cert
sudo mkdir /etc/apache2/ssl
sudo make-ssl-cert /usr/share/ssl-cert/ssleay.cnf /etc/apache2/ssl/apache.pem
NOTE: choose username is localhost). make-ssl-cert requires a source template (…/ssleay.cnf) and export the certificate to a file (…/apache.pem)
5. Create/Enable a site:
With Apache2 installed, you now have to create an URL (a sitename) and then enable it so that user can access to this site. This site, in this case, will later be used to redirect to the SVN repository and whenever user open this URL, it will display the directory structure of the repository.
A Web server like Apache2, of course, should be able to host multiple websites on a single machine. This can be done via virtual hosting (VirtualHost directive). Inside this directive, it contains enough information linking to the physical location of the site (e.g. Directory (where the site is), ServerAdmin (email of person in charge), ServerAlias (URL), port). By this way, you can access to your local repository using an URL other than localhost.
By default, each project will has a separate config file containing its VirtualHost directive. Apache2 will looks for them in the location /etc/apache2/sites-enabled. The name of the config file (/mylaptop) is also the relative path to the site (www…/mylaptop).
The default information for a virtual host is stored in /etc/apache2/sites-available/default or …/default-ssl. Here, you want to configure for a new site (i.e. site mapping a domain name (URL) to a physical SVN repository), we create a new file by making a copy of one of these two default files and modify it. Since we want to create a domain name that can be accessed via SSL, we clone default-ssl
sudo cp /etc/apache2/sites-available/default-ssl /etc/apache2/sites-available/$SITENAME
with $SITENAME is the name of the new site (e.g. repository)
Now add the following lines to the $SITENAME file. Suppose we want the domain name is dev.repository.com
|-(add)-> NameVirtualHost *:443|
|<VirtualHost _default_:443>||-(to)-> <VirtualHost *:443>|
|-(add)-> ServerName http://www.dev-repos.org|
|D ocumentRoot /var/www||-(keep)-> DocumentRoot /var/www|
add/uncomment these lines:
The first line (SSLEngine) tell Apache2 server to use SSL connection for this site. The second line means that we use the generated certificates for identification user-end.
Now, you have to tell the DNS which machine does http://www.dev-repos.org maps to (which IP). To do so, you can modify the file /etc/hosts and add these lines
127.0.0.1 localhost.localdomain localhost
If you want to access this URL from other machine, 127.0.0.1 should be substituted by the real IP of the SVN-server machine.
REMARK: This method is not convenient as you have to modify the hosts files in many machine. The more convenient way is to have a DNS server for the local network. We will discuss this faster way later.
Now, you restart Apache2.
sudo /etc/init.d/apache2 restart
You test if you can access the URL https://www.dev-repos.org from your browser (NOTE: https:// is for SSL connection). No, you cannot access to it. The answer is that you have to enable it.
To enable a new domain (a URL), e.g. for it to be accessible, you need to use a2ensite command (apache2 enable site).
sudo a2ensite $SITENAME sudo /etc/init.d/apache2 restart # or reload
with $SITENAME should be replaced by the one you’ve just created.
A warning about the self-signed certificate can be overcome by adding an exception for it.
- After this steps, the site https://www.dev-repos.org/ should work. However, there is nothing in this site.
- A warning that complaints about failure of server name determination can be fixed by adding
ServerName $SERVERNAMEto the main Apache config /etc/apache2/apache2.conf
6. Create repository(ies):
In the previous steps, you have created a site (URL) that enable us to connect to the real content. By default, the real content is stored in /var/www.
Now you will create the SVN repository where the version-control information are stored and then will map the URL (e.g. http://www.dev-repos.org) to it.
In this step we will install Subversion and create a folder for repository
$sudo apt-get install subversion
Once Subversion is installed, we have to create a repository where all workings files/folders can be imported for version control.
Read this section first to know
- what data should be added to the repos, ….
- use single repos for multiple project or a single repos for every project… With single repos, you have the advantage of doing single operation for all (e.g. do one thing for routinely back up all, one command to dump/load…) Single repos is used unless each project requires different event triggers, different mailing list…
- for each project (~ repos), create three subdirectories beneath the root of repository: trunk (where main project development occur), branches (where various named branches of the main development line), and tags (collection of tree snapshots that are created, or destroyed but never changed)
- which data storage mechanism to use (FSFS or BDB)
(1) Create a repos. Here, single repos – multiple projects is used
$sudo mkdir /var/svn
(2) Now, you want to create your first project in the repos
$sudo svnadmin create --fs-type fsfs /var/svn/$REPOS
replace $REPOS with your project’s name (e.g. firefox if you’re developing a software named firefox)
You are free to choose fsfs (FSFS) or bdb (Berkeley DB). I recommend fsfs.
NOTE: In SVN, we should now the following client-side commands: commit, import, export, checkout, add, info… while svnadmin and svnlook are both server-side commands.
A repository is a secure location, not any one can checkout or commit the data to it. We will create a group of users who are allowed to do this (e.g. persons who are working on the project whose codes are stored there):
Choose System > Administration > Users and Groups from your Ubuntu menu.
Select the Group tab
Click the ‘Add Group’ button
Name the group ‘subversion’
- At least, add yourself and www-data (the Apache user) as users to this group
(Note: in order to see www-data user, hits Alt-F2 to open Gconf-editor and modify root’s /apps/gnome-system-tools/users/showall)
- Add any other users you want
- Select ‘OK’ to commit your changes and exit the tool.
You have to logout and login again before the new setting takes effect. Next, you grant the group subversion access to the corresponding project (or the whole repository) (indicated by its path /var/svn/$REPOS or /var/svn only)
sudo chown -R www-data:subversion /var/svn/$REPOS
sudo chmod -R g+ws /var/svn/$REPOS
or (single repos - one project)
sudo chown -R www-data:subversion /var/svn/
sudo chmod -R g+ws /var/svn/
sudo chown -R www-data:www-data /var/svn/$REPOS
The last command sets gid for proper permissions on all new files added to your Subversion repository.
- /var /svn is a good place to store the repository since the folder /var can be accessed by other users.
- However, for single-user environment, I recommend to use ~/svn as the storage place. If we change this, remember to change the content of /etc/apache2/mods-available/dav_svn.conf –> (step 8) use this (
SVNPath /var/svn) instead of SVNParentPath:
7. Adding Basic Authentication:
At this point, you already have a repository that can be accessed by approved users. Now, you want to allow these users to access to the repos via web browser. Section 7&8 will help you do that. In this section, you are told to create a list of authorized users.
There are basically two ways to access to an SVN repos from web browser: as an Apache2 module ( mod_dav_svn module) or to use svnserve command line. The best solution is to use Apache2 for SVN (remember to restart Apache2 each time you enable a new service).
At this point, we already have a domain (a URL with empty content) and a physical location repository. To gain access to this repository via that domain, we need to enable dav_svn module.
sudo a2enmod dav sudo apt-get install libapache2-svn
You need to set-up again the list of users that can get access to the repository via web browser, we should create a password file to store a list of legal users (username + password).
sudo htpasswd -c -m /etc/apache2/dav_svn.passwd $AUTH_USER
with $AUTH_USER is the username who can access to the repository (e.g. your current username), you will be ased to give a corresponding password also.
- You will later tell Apache2 to use this lists for security whenever there is an access to the repository via web dav.
- dav_svn.passwd is just the file to store the password, you can choose whatever name you like.
- If you’re creating the file for the first time, use both options -c -m; otherwise, skip the -c (create) option when you want to add a new user to the repository’s access list to avoid overwriting the passwd file.
sudo htpasswd /etc/apache2/dav_svn.passwd second_user_name
8. Enable and configure WebDAV and SVN:
After creating a list of authorized users in section 7, now you can configure web dav.
(1) Open this file /etc/apache2/mods-available/dav_svn.conf and add (uncomment)
SVNParentPath /var/svn # should be compatible with that in step 6
AuthName "Subversion Repository"
- As mentioned in section 6, if a single repository is for every project, then you should use SVNPath instead of SVNParentPath.
- To allow non-anonymous access, you comment out:
#<LimitExcept GET PROPFIND OPTIONS REPORT>
Đây là thiết lập chung cho mọi sites (URL) có dùng web dav, nếu cấu hình riêng cho một site nào đó, thì có thể overwrite cấu hình mặc định này bằng cách lưu thiết lập mới vào VirtualHost, i.e. /etc/apache2/sites-available/$SITENAME.
- Apache won’t work until you open /etc/apache2/ports.conf, and remove the two lines: Listen 80, Listen 443
sudo /etc/init.d/apache2 restart
We can use SSL, open: https://$YOURHOST:443/svn/$REPOS
- The repository, of course now is empty. We need to add (import) projects into it.
11.Working with repository
11.1 Import data
Let’s SVN manage your data. Now, the important part is to import a project into SVN.
$svn import -m "First import to SVN" /path/to/the/project file:///path/to/the/repository
or you can use via the domain name.
$svn import -m "First import to SVN" /path/to/the/project https://dev.repository.com/svn/MyLaptop/project_name
$svn import -m "First import to SVN" /path/to/the/project file:///var/svn/MyLaptop/project/trunk
Without -m “….”, we have to add some comments associating with that import (just to remind user what we import). Now, we can access to the repository again by openning the link https://localhost/svn/MyLaptop
- There can be warning about the virtualhost
- To access the localhost, we now have to use https protocol
After the PI (project investigator) has done necessary setups for the project. To start working on that project, users have to checkout it to their working directory:
$ svn co https://localhost/svn/MyLaptop /var/www/your_working_directory --username user_name
where your_working_place is the place that you will ease the development process, i.e. where we want to make the copy and work with the files there. user_name is the user name of the person who can access the repository.
Commonly used commands:
svn checkout https://server.example.com/srv/svn/agaric myworkingcopyfolder
svn add yournewfilehere
svn add yournewdirectoryincludingfiles/ svn add * --force !add all files
svn commit -m "I made changes. Woohoo."
svnadmin allows you to perform several maintenance operations. It works via directo repository access, thus it can only be used on the machine that host the repository.
$ svnadmin help
svnadmin create # set up a new repository structure to an existing folder
svnadmin dump REPOS_PATH [-r LOWER[:UPPER]] [–incremental] > new_dump_file
# dump the content of the repos. from revision LOWER to the first (or UPPER). By default, it dumps only the first revision.
svnadmin hotcopy REPOS_PATH NEW_REPOS_PATH
# create a mirror copy of the current repos.
svnadmin load REPOS_PATH < new_dump_file
# load a repository dump stream from file (or stdin), and commit new revisions into the repository’s file system.
svnadmin upgrade REPOS_PATH
# upgrade the repos. to the latest support schema version (of BDB or FSFS)
“This is the tool for examining the various revision and transactions in a repository”….This tool can be used for diagnostic purpose. To work with revision we use –revision (-r) and with transaction we use –transaction (-t)
$svnlook info /var/svn/MyLaptop
# return: author \n date \n number of characters in the log message \n log message of the latest revision
$svnlook info /var/svn/MyLaptop -r 10 # info as above of the 10th revision
$svnlook youngest /var/svn/MyLaptop # print out the youngest revision number.
# compare files
We can use svndumpfilter to quickly and easily modify SVN repository history data.There are two subcommands with it: include or exclude
Once the data is stored in the repository, it is not easy to remove it away. What if you accidentally add some files/folders to the repos. and now you want to wipe them away.
With svndumpfilter, you can either give it a list of paths you wish to keep (include) or a list that you want to not keep) and then pipe such repository dump data through the filter.
To do so:
- dump your repository to a dump data (which is a human-readable representation that shows a list of changes to the versioned data in the repository over time)
svnadmin dump /var/svn/$REPOS > repos-dumpfile
with $REPOS is MyLaptop for my example and repos-dumpfile is just the name of the file containing dump data.
In the $REPOS folder, of course, there will be different files/subfolders.
- choose (filter) the one that you want to remove or keep (normally, we choose the files/subfolders to delete, since it’s easier)
svndumpfilter exclude $GARBAGEFOLDER < repos-dumpfile > new-dumpfile
in case you want to remove everything except $BRANCH2USE (i.e. create a new branch of repository), use the below command
svndumpfilter include $BRANCH2USE <repos-dumpfile > new-dumpfile
The result will be a stream of dump data that contains only the versioned paths you want to keep. If you create a new repository, edit this dump file (new-dumpfile), modify Node-path and Node-copyfrom-path headers.
The final step is to create a new repositories, and lead the dump file to this new one.
svnadmin create $NEW_REPOS
where $NEW_REPOS can be /var/svn/MyLaptop.
svnadmin load –ignore-uuid $NEW_REPOS < new-dumpfile
SVNSync provides all functions required for maintaining a read-only mirror of a SVN repository. In other words, it copy a repository to another one which is being kept as a mirror. We rarely use this function in daily work.
For complete reference, read.
Commits your work (changes) to the server so that others team member can checkout and see the changes.
svn commit file1 file2 -m “A message to tell something”
Also, to make sure you get the latest work from other team members, use the updates data from the server (get any commits from other users)
13. GUI for SVN client
14. Trac wiki (issues management)
We can use Track to manage the project. Read TRAC article.
mkdir -p /var/local/trac/trac.yourdomain.com
15. LDAP (new)
DĐây là một bổ sung mới của bài viết, LDAP giúp quản lí user một cách tập trung.
Cài đặt php5-ldap
sudo apt-get install php5-ldap phpldapadmin
Tạo file cấu hình /etc/apache2/conf.d/ldap
LDAPSharedCacheSize 200000 LDAPCacheEntries 1024 LDAPCacheTTL 600 LDAPOpCacheEntries 1024 LDAPOpCacheTTL 600 <Location /ldap-status> SetHandler ldap-status </Location>
Để dùng LDAP quản lí userssudo a2enmod authnz_ldap sudo a2enmod ldap
Chỉnh sửa file: /etc/apache2/mods-enabled/dav_svn.conf<Location /svn>
SVNParentPath /svn # cập nhật cho đúng
AuthName “Subversion Server”
Tạo file người dùng
developers = yourusername, nextusername
[ / ]
@developers = rw
* = r
[test:/] * = r me = rw