Vietnamen’s Weblog

Time, Chances, Diligence, Intelligence: which is the most important?

Source control (version control) in Ubuntu

with 3 comments

Apache + SVN (subversion) + web +

From step 1 to 5, we will set up Apache and enable SSL service that provide a secure access to the repository via Web interface (https). Also, other protocols we can use are svn:///, file:/// to get the access to your repository.

1. Install Apaches (LAMP):

Apache is the most commonly used Web Server, at least in Linux system. To install it, hits

$sudo taskel

and choose LAMP server

To manage php, you should install also phpmyadmin (choose Apache2 when install phpMyAdmin)

sudo apt-get install php5 libapache2-mod-php5 libapache2-mod-auth-mysql php5-mysql phpmyadmin

Open

$sudo aptitude

and install subversion-tools, libapache2-svn

NOTE:

  • To make sure Apache2 is properly installed, check: http://localhost/
  • In Apache2, the config file is nolonger httpd.conf, but /etc/apache2/apache2.conf

2. Enable SSL:

We use a2enmod command to enable an Apache2 module (to disable, we use a2dismod). Here, we enable two essential modules: mod_rewrite and mod_ssl

 sudo a2enmod rewrite
sudo a2enmod ssl

NOTE: If you can’t enable mod_rewrite, edit the file /etc/apache2/sites-available/default, and change AllowOverride None to AllowOverride All.

sudo /etc/init.d/apache2 restart
#sudo sh -c "echo 'Listen 443' >> /etc/apache2/ports.conf"

NOTE:

  • Port 443 is generally reserved for SSL

3. Generate Certificate:

A secure connection requires the two ends (server-side & client-side) to have some identification mechanism.The client can be anonymous to the server, yet the server need to be certificated.

One way to generate a self-signed certificate key on the server-side is to use make-ssl-cert program:

sudo apt-get install ssl-cert
sudo mkdir /etc/apache2/ssl
sudo make-ssl-cert /usr/share/ssl-cert/ssleay.cnf /etc/apache2/ssl/apache.pem

NOTE: choose username is localhost). make-ssl-cert requires a source template (…/ssleay.cnf) and export the certificate to a file (…/apache.pem)

5. Create/Enable a site:

With Apache2 installed, you now have to create an URL (a sitename) and then enable it so that user can access to this site. This site, in this case, will later be used to redirect to the SVN repository and whenever user open this URL, it will display the directory structure of the repository.

Create

A Web server like Apache2, of course, should be able to host multiple websites on a single machine. This can be done via virtual hosting (VirtualHost directive). Inside this directive, it contains enough information linking to the physical location of the site (e.g. Directory (where the site is), ServerAdmin (email of person in charge), ServerAlias (URL), port). By this way, you can access to your local repository using an URL other than localhost.

By default, each project will has a separate config file containing its VirtualHost directive. Apache2 will looks for them in the location /etc/apache2/sites-enabled. The name of the config file (/mylaptop) is also the relative path to the site (www…/mylaptop).

The default information for a virtual host is stored in /etc/apache2/sites-available/default or …/default-ssl. Here, you want to configure for a new site (i.e. site mapping a domain name (URL) to a physical SVN repository), we create a new file by making a copy of one of these two default files and modify it. Since we want to create a domain name that can be accessed via SSL, we clone default-ssl

sudo cp /etc/apache2/sites-available/default-ssl /etc/apache2/sites-available/$SITENAME

with $SITENAME is the name of the new site (e.g. repository)

Now add the following lines to the $SITENAME file. Suppose we want the domain name is dev.repository.com

<IfModule mod_ssl.c>
-(add)->      NameVirtualHost  *:443
<VirtualHost _default_:443> -(to)-> <VirtualHost *:443>
-(add)-> ServerName http://www.dev-repos.org
D ocumentRoot /var/www -(keep)->     DocumentRoot /var/www

add/uncomment these lines:

SSLEngine on
SSLCertificateFile /etc/apache2/ssl/apache.pem
SSLProtocol all
SSLCipherSuite HIGH:MEDIUM

The first line (SSLEngine) tell Apache2 server to use SSL connection for this site. The second line means that we use the generated certificates for identification user-end.

Now, you have to tell the DNS which machine does http://www.dev-repos.org maps to (which IP). To do so, you can modify the file /etc/hosts and add these lines

127.0.0.1 localhost.localdomain localhost

127.0.0.1 http://www.dev-repos.org

If you want to access this URL from other machine, 127.0.0.1 should be substituted by the real IP of the SVN-server machine.
REMARK: This method is not convenient as you have to modify the hosts files in many machine. The more convenient way is to have a DNS server for the local network. We will discuss this faster way later.

Now, you restart Apache2.

sudo /etc/init.d/apache2 restart

You test if you can access the URL https://www.dev-repos.org from your browser (NOTE: https:// is for SSL connection).  No, you cannot access to it. The answer is that you have to enable it.

Enable

To enable a new domain (a URL), e.g. for it to be accessible, you need to use a2ensite command (apache2 enable site).

sudo a2ensite $SITENAME
sudo /etc/init.d/apache2 restart # or reload

with $SITENAME should be replaced by the one you’ve just created.

A warning about the self-signed certificate can be overcome by adding an exception for it.

NOTE:

  • After this steps, the site https://www.dev-repos.org/ should work. However, there is nothing in this site.
  • A warning that complaints about failure of server name determination can be fixed by adding ServerName $SERVERNAME to the main Apache config /etc/apache2/apache2.conf

6. Create repository(ies):

In the previous steps, you have created a site  (URL) that enable us to connect to the real content. By default, the real content is stored in /var/www.
Now you will create the SVN repository where the version-control information are stored and then will map the URL (e.g. http://www.dev-repos.org)  to it.

Create

In this step we will install Subversion and create a folder for repository

$sudo apt-get install subversion 

Once Subversion is installed, we have to create a repository where all workings files/folders can be imported for version control.

Read this section first to know

  • what data should be added to the repos, ….
  • use single repos for multiple project or a single repos for every project… With single repos, you have the advantage of doing single operation for all (e.g. do one thing for routinely back up all, one command to dump/load…) Single repos is used unless each project requires different event triggers, different mailing list…
  • for each project (~ repos), create three subdirectories beneath the root of repository: trunk (where main project development occur), branches (where various named branches of the main development line), and tags (collection of tree snapshots that are created, or destroyed but never changed)
  • which data storage mechanism to use (FSFS or BDB)

(1) Create a repos. Here, single repos – multiple projects is used

$sudo mkdir /var/svn

(2) Now, you want to create your first project in the repos

$sudo svnadmin create  --fs-type fsfs /var/svn/$REPOS

replace $REPOS with your project’s name (e.g. firefox if you’re developing a software named firefox)

You are free to choose fsfs (FSFS) or bdb (Berkeley DB). I recommend fsfs.


NOTE: In SVN, we should now the following client-side commands: commit, import, export, checkout, add, info… while svnadmin and svnlook are both server-side commands.

Gain access

A repository is a secure location, not any one can checkout or commit the data to it. We will create a group of users who are allowed to do this (e.g. persons who are working on the project whose codes are stored there):

  1. Choose System > Administration > Users and Groups from your Ubuntu menu.

  2. Select the Group tab

  3. Click the ‘Add Group’ button

  4. Name the group ‘subversion’

  5. At least, add yourself and www-data (the Apache user) as users to this group
    • (Note: in order to see www-data user, hits Alt-F2 to open Gconf-editor and modify root’s /apps/gnome-system-tools/users/showall)

    • Add any other users you want
  6. Select ‘OK’ to commit your changes and exit the tool.

You have to logout and login again before the new setting takes effect. Next, you grant the group subversion access to the corresponding project (or the whole repository) (indicated by its path /var/svn/$REPOS or /var/svn only)

sudo chown -R www-data:subversion /var/svn/$REPOS
sudo chmod -R g+ws /var/svn/$REPOS

or (single repos - one project)

sudo chown -R www-data:subversion /var/svn/
sudo chmod -R g+ws /var/svn/

sudo chown -R www-data:www-data /var/svn/$REPOS

The last command sets gid for proper permissions on all new files added to your Subversion repository.

NOTE:

  • /var /svn    is a good place to store the repository since the folder /var can be accessed by other users.
  • However, for single-user environment, I recommend to use ~/svn as the storage place. If we change this, remember to change the content of /etc/apache2/mods-available/dav_svn.conf    –> (step 8) use this (SVNPath /var/svn) instead of SVNParentPath:


7. Adding Basic Authentication:

At this point, you already have a repository that can be accessed by approved users. Now, you want to allow these users to access to the repos via web browser. Section 7&8 will help you do that. In this section, you are told to create a list of authorized users.

There are basically two ways to access to an SVN repos from web browser: as an Apache2 module ( mod_dav_svn module) or to use svnserve command line. The best solution is to use Apache2 for SVN (remember to restart Apache2 each time you enable a new service).

At this point, we already have a domain (a URL with empty content) and a physical location repository. To gain access to this repository via that domain, we need to enable dav_svn module.

sudo a2enmod dav
sudo apt-get install libapache2-svn

You need to set-up again the list of users that can get access to the repository via web browser, we should create a password file to store a list of legal users (username + password).

sudo htpasswd -c -m /etc/apache2/dav_svn.passwd $AUTH_USER

with $AUTH_USER is the username who  can access to the repository (e.g. your current username), you will be ased to give a corresponding password also.

  • You will later tell Apache2 to use this lists for security whenever there is an access to the repository via web dav.

NOTE:

  • dav_svn.passwd is just the file to store the password, you can choose whatever name you like.
  • If you’re creating the file for the first time, use both options -c -m; otherwise, skip the -c (create) option when you want to add a new user to the repository’s access list to avoid overwriting the passwd file.
 sudo htpasswd /etc/apache2/dav_svn.passwd second_user_name

8. Enable and configure WebDAV and SVN:

After creating a list of authorized users in section 7, now you can configure web dav.

(1) Open this file /etc/apache2/mods-available/dav_svn.conf and add (uncomment)

<Location /svn>

DAV svn
SVNParentPath /var/svn # should be compatible with that in step 6
AuthType Basic
AuthName "Subversion Repository"
AuthUserFile /etc/apache2/dav_svn.passwd
Require valid-user
SSLRequireSSL

</Location>

NOTE:

  • As mentioned in section 6, if a single repository is for every project, then you should use  SVNPath instead of SVNParentPath.
  • To allow non-anonymous access, you comment out:

#<LimitExcept GET PROPFIND OPTIONS REPORT>
#</LimitExcept>

Đây là thiết lập chung cho mọi sites (URL) có dùng web dav, nếu cấu hình riêng cho một site nào đó, thì có thể overwrite cấu hình mặc định này bằng cách lưu thiết lập mới vào VirtualHost, i.e. /etc/apache2/sites-available/$SITENAME.

NOTE:

  • Apache won’t work until you open /etc/apache2/ports.conf, and remove the two lines: Listen 80, Listen 443

9. Finalization:

sudo /etc/init.d/apache2 restart

10. Test

We can use SSL, open: https://$YOURHOST:443/svn/$REPOS

https://localhost/svn/$REPOS

https://dev.repository/svn/$REPOS

https://www.dev-repos.org/svn/firefox

NOTE:

  • The repository, of course now is empty. We need to add (import) projects into it.

11.Working with repository

11.1 Import data

Let’s SVN manage your data. Now, the important part is to import a project into SVN.

$svn import -m "First import to SVN" /path/to/the/project    file:///path/to/the/repository

or you can use via the domain name.

$svn import -m "First import to SVN" /path/to/the/project    https://dev.repository.com/svn/MyLaptop/project_name

Example:

$svn import -m "First import to SVN" /path/to/the/project    file:///var/svn/MyLaptop/project/trunk

Without -m “….”, we have to add some comments associating with that import (just to remind user what we import). Now, we can access to the repository again by openning the link https://localhost/svn/MyLaptop

NOTE:

  • There can be warning about the virtualhost
  • To access the localhost, we now have to use https protocol

11.2. Checkout

After the PI (project investigator) has done necessary setups for the project. To start working on that project, users have to checkout it to their working directory:

$ svn co https://localhost/svn/MyLaptop /var/www/your_working_directory --username user_name

where your_working_place is the place that you will ease the development process, i.e. where we want to make the copy and work with the files there. user_name is the user name of the person who can access the repository.

12. Subversion

Commonly used commands:

svn checkout https://server.example.com/srv/svn/agaric myworkingcopyfolder
svn status
svn update
svn add yournewfilehere
svn add yournewdirectoryincludingfiles/
svn add * --force            !add all files
svn commit -m "I made changes.  Woohoo."

Server-side commands

For repository maintenance:

SVNAdmin

svnadmin allows you to perform several maintenance operations. It works via directo repository access, thus it can only be used on the machine that host the repository.

$ svnadmin help

svnadmin create         # set up a new repository structure to an existing folder

svnadmin dump REPOS_PATH [-r LOWER[:UPPER]] [–incremental] > new_dump_file

# dump the content of the repos. from revision LOWER to the first (or UPPER). By default, it dumps only the first revision.

svnadmin hotcopy REPOS_PATH NEW_REPOS_PATH

# create a mirror copy of the current repos.

svnadmin load REPOS_PATH  < new_dump_file

# load a repository dump stream from file (or stdin), and commit new revisions into the repository’s file system.

svnadmin upgrade REPOS_PATH

# upgrade the repos. to the latest support schema version (of BDB or FSFS)

SVNLook

$svnlook help

“This is the tool for examining the various revision and transactions in a repository”….This tool can be used for diagnostic purpose. To work with revision we use –revision (-r) and with transaction we use –transaction (-t)

$svnlook info /var/svn/MyLaptop

# return: author \n  date \n number of characters in the log message \n log message of the latest revision

$svnlook info /var/svn/MyLaptop -r 10 # info as above of the 10th revision

$svnlook youngest /var/svn/MyLaptop # print out the youngest revision number.

svnlook diff

# compare files

SVNDumpFilter

$svndumpfilter help

We can use svndumpfilter to quickly and easily modify SVN repository history data.There are two subcommands with it: include or exclude

Once the data is stored in the repository, it is not easy to remove it away. What if you accidentally add some files/folders to the repos. and now you want to wipe them away.

With svndumpfilter, you can either give it a list of paths you wish to keep (include) or a list that you want to not keep) and then pipe such repository dump data through the filter.

To do so:

  • dump your repository to a dump data (which is a human-readable representation that shows a list of changes to the versioned data in the repository over time)

svnadmin dump /var/svn/$REPOS > repos-dumpfile

with $REPOS is MyLaptop for my example and repos-dumpfile is just the name of the file containing dump data.

In the $REPOS folder, of course, there will be different files/subfolders.

  • choose (filter) the one that you want to remove or keep (normally, we choose the files/subfolders to delete, since it’s easier)

svndumpfilter exclude $GARBAGEFOLDER < repos-dumpfile > new-dumpfile

in case you want to remove everything except $BRANCH2USE (i.e. create a new branch of repository), use the below command

svndumpfilter include $BRANCH2USE <repos-dumpfile > new-dumpfile

The result will be a stream of dump data that contains only the versioned paths you want to keep. If you create a new repository, edit this dump file (new-dumpfile), modify Node-path and Node-copyfrom-path headers.

The final step is to create a new repositories, and lead the dump file to this new one.

svnadmin create $NEW_REPOS

where $NEW_REPOS can be /var/svn/MyLaptop.

svnadmin load –ignore-uuid $NEW_REPOS < new-dumpfile

Read more: http://svnbook.red-bean.com/en/1.5/svn.reposadmin.maint.html#svn.reposadmin.maint.filtering

SVNSync

$svnsync help

SVNSync provides all functions required for maintaining a read-only mirror of a SVN repository. In other words, it copy a repository to another one which is being kept as a mirror. We rarely use this function in daily work.

Client-side commands

For complete reference, read.

Commits your work (changes) to the server so that others team member can checkout and see the changes.

svn commit file1 file2 -m “A message to tell something”

Also, to make sure you get the latest work from other team members, use the updates data from the server (get any commits from other users)

svn up[date]

References:

  1. http://alephzarro.com/blog/2007/01/07/installation-of-subversion-on-ubuntu-with-apache-ssl-and-basicauth/
  2. https://help.ubuntu.com/community/Subversion
  3. http://ubuntuforums.org/archive/index.php/t-451986.html
  4. http://support.mozilla.com/tiki-view_forum_thread.php?locale=ca&comments_parentId=165672&forumId=1
  5. https://help.ubuntu.com/8.04/serverguide/C/subversion.html
  6. http://vegastrike.sourceforge.net/wiki/HowTo:Checkout_SVN#Ubuntu
  7. http://svnbook.red-bean.com/en/1.5/svn-book.html#svn.tour.initial
  8. http://robertbasic.com/blog/lamp-and-svn-on-ubuntu-8-10/
  9. http://www.debuntu.org/2006/02/22/7-virtual-hosting-using-apache-2
  10. http://svnbook.red-bean.com/en/1.5/svn.reposadmin.maint.html#svn.reposadmin.maint.filtering

13. GUI for SVN client

http://code.google.com/p/nautilussvn/

http://www.syntevo.com/smartsvn/index.html

14. Trac wiki (issues management)

We can use Track to manage the project. Read TRAC article.

mkdir -p /var/local/trac/trac.yourdomain.com

Read:

  1. http://www.voipphreak.ca/2008/11/02/tutorial-on-how-to-install-apache-subversion-and-trac-with-virtual-hosting-suppor-in-ubuntu-linux/
  2. http://trac.edgewall.org/wiki/TracOnDebian

15. LDAP (new)

DĐây là một bổ sung mới của bài viết, LDAP giúp quản lí user một cách tập trung.

Cài đặt php5-ldap

sudo apt-get install php5-ldap phpldapadmin

Tạo file cấu hình /etc/apache2/conf.d/ldap

LDAPSharedCacheSize 200000
LDAPCacheEntries 1024
LDAPCacheTTL 600
LDAPOpCacheEntries 1024
LDAPOpCacheTTL 600

<Location /ldap-status>
SetHandler ldap-status
</Location>

Để dùng LDAP quản lí users

sudo a2enmod authnz_ldap
sudo a2enmod ldap

Chỉnh sửa file: /etc/apache2/mods-enabled/dav_svn.conf

<Location /svn>

DAV svn

SVNParentPath /svn  # cập nhật cho đúng
AuthType Basic
AuthName “Subversion Server”
AuthLDAPEnabled on
AuthLDAPAuthoritative on
AuthLDAPURL ldap://localhost/ou=Users,dc=example,dc=com
#AuthzSVNAccessFile /etc/svn-users
AuthUserFile /etc/apache2/dav_svn.passwd
Require valid-user
</Location>

Tạo file người dùng /etc/svnaccess

[groups]
developers = yourusername, nextusername
[ / ]
@developers = rw
* = r
[test:/]
* = r
me = rw

Written by vietnamen

Tháng Hai 24, 2009 lúc 12:39 sáng

3 phản hồi

Subscribe to comments with RSS.

  1. […] việc dùng SVN, ta có thể dùng một tiện ích nhỏ gọn và tiện lợi cho emacs, đó là […]

  2. Hi there thanks for the cool article, it was helpful indeed. Where is the article on Trac? (page not found when clickin your link)

    CHeers,
    g

    virtualTrader

    Tháng Mười 14, 2009 at 2:06 sáng

    • i will make it available soon, thanks for visiting my page.🙂

      vietnamen

      Tháng Mười 18, 2009 at 10:52 sáng


Gửi phản hồi

Mời bạn điền thông tin vào ô dưới đây hoặc kích vào một biểu tượng để đăng nhập:

WordPress.com Logo

Bạn đang bình luận bằng tài khoản WordPress.com Log Out / Thay đổi )

Twitter picture

Bạn đang bình luận bằng tài khoản Twitter Log Out / Thay đổi )

Facebook photo

Bạn đang bình luận bằng tài khoản Facebook Log Out / Thay đổi )

Google+ photo

Bạn đang bình luận bằng tài khoản Google+ Log Out / Thay đổi )

Connecting to %s

%d bloggers like this: