Sublime Subversion

As you may know, last week saw the Subversion Vision Conference in New York City. This wasn’t a large event, rather it was a small meeting of several core members of the Subversion team. Their goal wasn’t marketing, community building, or flag waving, but rather to step back a moment and take a look at where Subversion was headed. And although they didn’t come away with a finalized roadmap or vision statement, it does appear that some good progress was made on mapping out the direction for Subversion.

I think this is great and I’m glad to see the Subversion team taking some time to map out the future. But I do wish we had seen some indication that Subversion may evolve to incorporate some distributed functionality. However, according to Michael Pilato the immediate plan appears to be to “Maintain a centralized approach to version control”.

Considering the composition of the team – most members being from companies such as Collabnet or WANdisco who serve the enterprise – this isn’t exactly surprising. Centralization is still a key concern for enterprise IT organizations who need to maintain tight control on their systems for compliance and governance reasons. But I hate to see the team dismiss different approaches to source control simply because it doesn’t appeal to the enterprise yet.

I think distributed version control systems have some great advantages and not just in the open source space. I think cheap, personal branching and merging is an extremely powerful tool. From personal experience I can tell you that it can completely change your development workflow for the better. My opinion is that anything that stops a developer from committing changes is a bad thing. Your version control system is your safety net – it should be there to support you, not to impose speed bumps.

I’ve watched the rise in popularity of DVCSs over the last few years. Git made the DVCS sexy but they’ve been around for quite a while. Microsoft supporting Mercurial on Codeplex is just one example of DVCSs being taken seriously by very large players.

But even though distributed systems are the hot “new” thing, most organizations – big or small – still place a high value on centralization. As a small business using Subversion, I see the value in having one location with all of our valuable IP. I have one asset to back up, one place to go to find repositories, one place to go to manage access, etc. These are all great arguments for a centralized system but I think there can be a middle ground.

The combination of a centralized version control system supporting distributed working copies could be extremely powerful. Allowing your developers to commit, branch, merge, and revert locally and then pushing changes up to the master repository as appropriate seems like the best of both worlds.

This is where the Git users out there yell at me and tell me that Github already works this way. I’m aware of that and have used it myself. It’s a great workflow – but it’s not ready for the enterprise, or even smaller companies. It will work for some, but not for the majority.

This is where I think the Subversion team can make some serious traction – not just supporting the status quo – but really moving the product forward in a new and exciting way. I think that Subversion is in the unique position of being able to take advantage of the benefits that distributed systems offer while at the same time serving those who care about centralization.

In the coming months I expect that the Subversion team will make their roadmap public and I’m excited to see what’s on it. I really like what I’ve been hearing about the Working Copy changes (no more .svn folders littering my working copy!) and love to see the momentum in the community. Subversion is still hands-down the best source control solution for small, medium and enterprise organizations, but it can always be better. I hope that the core team will consider some of these opportunities when mapping out Subversion’s future.

Over the last couple of days we have started posting some short tutorial videos. Initially these videos focus on how to use Sublime but over time we will expand the scope to cover Subversion topics as well.

• Tutorial 1: Creating Subversion Repositories in Sublime
• Tutorial 2: Checking out a Repository

Please let us know if these videos are helpful and if you have any suggestions on features to cover or ways we can improve them.

Apologies for the radio silence recently.  While finishing up Sublime 1.1 we decided to completely re-write the Sublime installer.  We had been using an off-the-shelf product for our installer which worked okay initially.  However, after digging into more of the details we decided to switch to the excellent open-source Wix installer.  While we believe this is a good long term decision, it did add quite a bit of development time to 1.1.  We’re working the last bugs out of the installer now and should have 1.1 released soon.

So what’s new in 1.1?  Well the biggest new feature is RSS Feeds.  With Sublime 1.1, each repository will expose an RSS feed of the commit history.  This can be a great alternative to email-based commit notifications.  Each item in the RSS feed will show the time of the commit, the author, the commit message, and the list of changed files.

In addition to repository RSS feeds we added an RSS feed for administrators.  This feed will contain important information about the server such as the success or failure of repository backups, information about new users or repositories created, etc.

We’ve been using both of these features on our internal servers for a couple of months and really like them.  Hopefully you will two.

In addition to those features we’ve included a number of minor enhancements and bugfixes including:

• Turn On/Off Repository folder hierarchy.  This allows administrators to specify whether repositories will be created in a folder hierarchy or a flat list.
• Better support for non-English Subversion servers
• UI bug fixes including better paging support for repositories with a large number of commits
• Email notifications enabled by default for new repositories

Thanks to all of you who have sent us feedback and suggestions.  Once 1.1 is released we’ll post our plans for 1.2.  Remember, 1.x releases of Sublime are free upgrades to anybody who has purchased a Sublime license.

The Board of the Apahce Software Foundation (ASF) has voted to approve the graduation of Subversion from incubation.  Subversion is now a full-fledged ASF project.  This is just one more step in the life of Subversion and one that will hopefully encourage greater community involvement and innovation.

Even if you are new to Subversion you have likely heard terms like “trunk”, “branching” and “tags” thrown around.  If you are coming from a source control system like CVS you may already be familiar with these terms but if you are coming from a tool like Visual Source safe they may be new.  This post provides an introduction to these concepts in Subversion.  For more in-depth information please see Chapter 4 of the Subversion Book.

First things first – the trunk.  In Subversion, the term trunk refers to your main source code tree.  If you are starting out a new project, you will spend the majority of your time making changes and committing them to the trunk of your repository.  This is because early in a project there are fewer situations where you need branches or tags (there are certainly exceptions depending on your team and development practice, but this is an introductory article and we won’t get into that).

Now, let’s say your project is moving along and getting fairly stable, but you want to experiment with adding a new feature.  You could simply start making changes in your working copy and see how they work out without ever committing them, but this would be a mistake.  If this is a large set of changes your work could span multiple days – even weeks.  At some point your manager is going to wander over to your desk and inform you that there is a critical change that needs to be made RIGHT NOW. Now you’ll need to set aside those experimental changes, get a fresh copy of the trunk, commit those, somehow port those changes into your experimental code, etc etc.  It gets complicated fast.

Branching is specifically designed for situations like this.  A branch is a copy of your source code – typically a copy of the trunk – with a special name like “experimental feature”.  It sets up a space for you to work and make changes without committing to the main trunk.  Once the code in your branch is complete, you can merge your changes back to the trunk.  In addition, as new changes are committed to the main trunk, you can merge those changes into your branch so your branch is up to date with the latest changes from the trunk.  This is a great habit to get into by the way because it makes the process of merging your branch back to the trunk much much easier when the time comes.  Not to mention the fact that you are much more likely to catch potential integration issues early.

I’ll come back to branching in a bit, but now let’s move to tagging.

A tag is simply a named version of your code at a particular point in time.  For example, let’s say you’ve reached version 1.0 of your project.  Rather than having to remember that version 1.0 was revision 638, you can create a tag and give it the name of “1.0″.  Best practice is to create a tag for every major release or milestone (beta, 1.0, 1.1, 2.0, etc) but you can create tags for any point in time you think developers will care about in the future.

So we’ve covered the basic concepts, now I’m going to let you in on a little secret.  Ready?  In Subversion, there is literally nothing special about a branch, or a tag, or the trunk for that matter.  It’s just a directory in your repository which holds a version of your source code.  That’s it.  In most Subversion repositories the directory for storing branches is called, well “branches”, but that is just a naming convention.  You could call yours “superman” and Subversion would still behave exactly the same.

Here’s what I mean.  Most repositories will have the following structure at their root:
/branches
/tags
/trunkEach path above is simply a directory in your repository.  Trunk is self explanatory – it contains your main development tree.  Branches is a folder which will contain sub-folders, one for each branch.  Tags is the same – a folder which will contain sub-folders for each tag.

Here’s what your repository might look like with several tags and branches:

/branches
/branches/new_feature <- branch for implementing a new feature
/branches/1.0_maint <- branch for doing maintenance on version 1.0
/tags
/tags/0.9 <- version 0.9 release
/tags/1.0 <- version 1.0 release
/trunkWhen you create a branch, or a tag, Subversion is simply copying your current code usually from the trunk (although not required) into a new folder in /branches or /tags. Once your code is copied, you can check out a new working copy from that location, make changes, commit, etc. Since your working copy points at the branch folder rather than the trunk, changes you make will only be committed to the branch and not to the trunk. The act of merging is essentially picking the changes you have made and moving them back into the trunk. Subversion gives us tools to make this easier, but it essentially boils down to just that.

Now, why is this important? Well for one it means you aren’t stuck with the standard naming convention of branches, tags, and trunk. If you prefer something like “variations”, “labels”, and “root” it is entirely up to you – Subversion won’t care. However, unless you have a compelling reason to change, it’s best to stick with branches, tags, and trunk because it is widely accepted as the standard naming convention.

Another reason this is important is that Subversion isn’t going to limit you from making mistakes like checking out a tag and making changes to it. Making changes to a tag goes against the entire purpose of a tag because a tag should be a static point in the history of your code – not something you make changes to. If you need to make changes to version 1.0 outside of your trunk (maintenance for example) you should make a new branch using your 1.0 tag as the source (step by step instructions is beyond the scope of this article).

So that’s it. Branches, tags, and trunk are simply standard folders in your repository set up to help you manage your development process. You can change these conventions if you wish, but you should have a solid reason for doing so. You can also add to this structure if you need additional controls for your development team or process.

Before closing I want to make one point about how your code is copied to a branch or a tag. Subversion is smart enough not to make an entire duplicate copy of your code each time you create a branch or tag. It simply creates a pointer to the revision your branch is based on. When you make a commit to the branch, Subversion stores the difference between the modified version and the original file. It does not create a full copy of the file.

Over the coming months we will be releasing the roadmap for Sublime. This will include new features and updates for version 1.0 (which are available free to all current 1.0 users) as well as future plans. To make sure we are planning the right features for Sublime we have set up a user forum to capture your feedback and ideas. Here you can vote on ideas submitted by other users, or submit your own suggestions.

Currently the top contenders are:

• SharePoint integration – creating a repository will create or connect to a SharePoint project site
• Repository History RSS feed – expose an RSS feed on repository commits – may be used as an alternative to email announcements
• Path-based permissions – allow permissions to be set on paths within a repository
• Active Directory Groups – allow AD security groups to be used when assigning permissions

There are plenty of other ideas as well ranging from an enhanced repository browser, incremental backups, and security/permission audit.

Please help us make Sublime the best Subversion server available by providing your input and ideas!

Sublime will automatically send email notifications to your developers after a commit is made.  You can enable email notifications for your repositories by navigating to the Repository in Sublime, clicking the Edit Settings link, and changing the Email Notification setting to “On”.

By default, each email notification looks like this:

Default subversion email notification

However, this can be customized by editing an HTML template file located on your Sublime subversion server.

1. Connect to your server using Remote Desktop
2. Locate the “bin” folder inside your Sublime installation directory (C:\Program Files\Sublime\bin by default).
3. Make a backup copy of the file “EmailTemplate.htm”.  This is optional, but recommended so that you will be able to revert to the default template if you wish.
4. Now, using a text editor such as notepad, open “EmailTemplate.htm”.
5. Modify the HTML to customize the body of the email message.  When Sublime sends an email notification, it replaces certain placeholder tokens in the email template with actual commit details.  For example, “{REPOS}” is replaced with the name of the repository.  A complete list of placeholder tokens is listed below.
6. Save “EmailTemplate.htm”.

Here’s an example of a customized email template.  We have added our company logo at the top, and excluded some of the commit details.

Customized email notification

Placeholder Token Reference

The table below contains the complete list of placeholder tokens which may be used in your email notification template. The example for each is taken from the screen shot below the table so you can see exactly where the token is used in the email notification.

Default subversion email notification

I came across this article today in another forum.  It’s a translation of a Russian article about how the source code for websites that use Subversion can be obtained if they don’t remove or block the hidden .svn folders from their site structure.

Basically, since Subversion maintains a text-based copy of your files in the hidden .svn directory, it’s possible to view the source directly if you know that a given website is using Subversion.

For example, if you want to view the source to index.php on a given site, you could try a URL like this:
mysite.com/.svn/text-base/index.php.svn-base
The server would see that as a request for a .svn-base file, not a .php file and it would simply serve up the source rather than running through PHP.

There are two ways to protect you against this.  First, as I mentioned in a previous post, DON’T DEPLOY .svn FOLDERS.  The svn export command lets you grab a full tree of your source code without the .svn folders – so use that if you are deploying a website.

Second, you can set up your web server to block .svn folders.  The article has some examples of how to do that for apache and nginx.  I haven’t tried this on IIS yet but I’ll give it a shot and update this post with steps to protect yourself on Windows.

Sublime automatically backs up all of your Subversion repositories on a schedule you define.  In this short article we will walk through the process of configuring your backups, monitoring your server to ensure backups are happening, and restoring a repository.

1. Configuring Backups

To configure your backups, navigate to the Administration section in Sublime and click the “Backups” tab.  First, enter a path for your repositories to be backed up to.  When you install Sublime, it will default this path to be a Backup folder within your Sublime installation directory.  However, it is strongly recommended that you back your repositories up to a network device such as a NAS or remote network share.

Next, enter a time of day and the days of the week that your backups should occur.  Generally this should be a time of the day that users are not using Subversion.  However, Sublime uses the subversion “hotcopy” command when making backups so even if your users are still accessing Subversion you are not at risk of corrupting data or interrupting the backups.

Lastly, enter a backup account to be used when performing the backups.  This account must have read/write access to the Backup path.  It will also be added to the local administrators group on your subversion server.

Scheduling subversion backups in Sublime

2. Monitoring Backups

Each time Sublime backs up your Subversion repositories, it will write an entry to the Windows Event Log.  You can monitor this directly from the Sublime web interface, or by connecting to the server using remote desktop.  Each entry in the windows event log will list the number of repositories backed up, the number of errors (if any), and the duration for the backup.

Successful Subversion backup in the Windows Event Log

If you are using Windows Server 2008, you can create a Custom View to show only Subversion backup messages.  Simply create a new Custom View to display “Information” events filtered where the Source is “Sublime.SubMaint”.

3. Restoring a Subversion Repository

Restoring a repository is essentially copying the backup repository from the backup location back to your Subversion server.  However, as a best practice we recommend the following steps:

1. Take your Subversion service offline.  This ensures that while the restore is underway, your developers cannot commit to the repository.  If you are running Sublime in SVNSERVE mode you can take the Subversion service offline from the administration home page.  If you are using Apache you will have to stop the service manually in the Services console in Windows.
2. Locate the folder containing the repository you want to restore on both your Subversion server, and in the backup location.  The backup location should be a mirror of your repositories folder on your Subversion server, so locating the correct repository should be easy (see below for tips on identifying a repository folder).
3. On your subversion server, delete or rename the folder containing the repository you want to restore.  Then, copy the repository folder from the backup location over to your subversion server.

Let’s look at an example.  In the screen shot below, the left folder is the Repositories folder on our Subversion server.  This contains all of our current “live” repositories.  The folder on the right is our backup location.  If we wanted to restore the “CommonLibs” repository, first we would delete “CommonLibs” from the “Archive” folder on the left.  Then we would copy the “CommonLibs” folder from the backup location on the right and paste it in the “Archive” folder on the left.  The result would be a fully restored version of the “CommonLibs” repository from the time the last backup ran.

The Backup location is a mirror of your Repository location

Once you have restored your repository you can bring the Subversion service back online.  It is also a good idea to reset IIS (open a command prompt and type “iisreset”) to ensure that the Sublime cache is cleared.

TIP: How to locate a repository folder

If you are new to Subversion it may not be clear exactly what constitutes a repository.  In short, a repository consists of a folder containing a number of specific sub-folders and files.  So when we say “repository” we are talking about the parent folder and all child folders and files.

You can identify the repository parent folder by looking at its children.  A repository will contain the following child folders: conf, db, hooks, locks.  It will also contain a file called “format” and there may or may not be a README.txt.  In the following screen shot, we are looking at the content of “CommonLib”  You can see that it contains these folders, thus “CommonLib” is a repository.  The parent folder “Archive” is not – it’s merely a folder.

Identifying a repository folder

Repository Templates are a great way to reduce the time required for setting up a new project, and encourage consistency across projects.  Consistency is important especially in an organization with many different repositories spread over different projects and development teams.  Having a consistent repository structure drastically reduces common errors made by new developers or even experienced developers in an unfamiliar repository.

Setting up Repository Templates in Sublime is as easy as creating a folder structure on your Sublime server containing the files and folders you want to include in your new project.  In this walk-through we’re going to create a Repository Template for new projects using Sharp Architecture.  Sharp Architecture is a perfect candidate for a Repository Template because there are quite a few dependencies that can take a while to download and organize.

In the following steps we’ll be creating a Repository Template for Sharp Architecture, but the same steps can be applied to any number of project types you may have.

1. Create the new Repository Template Folder

The first step is to create the new Repository Template folder on your Sublime/Subversion server.

1. Connect to your server using Remote Desktop
2. Navigate to the “Templates” folder beneath your Sublime install location (typically C:\Program Files\Sublime\Templates)
3. Create a new folder inside Templates called “SharpArch” (note: the folder name can be different from what your users see when choosing a template)
   

   Create a new folder for the template

4. Open the new folder and create the standard Subversion trunk, branches, and tags folders.
   

   Create the branches, tags, and trunk folder

5. Open the “trunk” folder you just created.  Now it’s time to start creating the folder structure for your new project.  For starters, Sharp Architecture projects tend to have a defined top-level folder structures for the different elements of a project: app, build, db, tests, etc.  So let’s create those folders now.
   

   Create the project folder structure

6. Next, if you like you can create README.txt files in each folder explaining what should be placed there.  This is optional of course but can be useful for new developers who may not be familiar with the project structure.

2. Download Sharp Architecture binaries

Now that our basic structure has been created, it’s time to download the related Sharp Architecture binaries as well as their dependencies.

1. Open a web browser and navigate to the Sharp Architecture download page.
2. Download and extract the latest revision (or which ever revision you prefer) of the compiled binaries
3. In the extracted folder, locate all of the DLLs and associated files (located within the “bin” folder as of this writing) and copy them all to the “lib” folder we created back in step 5 of the previous section.
   

   Copy Sharp Architecture binaries to template

3. Register the Template with Sublime

The last step is to register the newly created Repository Template with Sublime.

1. Navigate back to the Templates folder and open “templates.txt” in Notepad
2. This file follows a simple INI file format and you should have two templates already listed (Standard and Empty).  Create a new section with the following structure:
   [SharpArch]
name = Sharp Architecture
description = Use for Sharp Architecture based projects.
   

   Insert new template description into templates.txt

3. In the example above, the section header “[SharpArch]” must be the same as the folder name for the template.  But the value for the “name” attribute is what will be displayed to your users.  There is also an optional “default = true” attribute you can add if you want this template to be automatically selected as the default choice when creating new repositories.
4. Save templates.txt and close Notepad.
5. Next you must restart IIS by opening a command prompt and typing
   C:\> iisreset

4. Test the Template

Open a web browser and navigate to Sublime.  Click on the Create New Repository tab and in the Template section, you should see the new “Sharp Architecture” template.

Preview the new template

Wrap-up

In this walkthrough we’ve created a new Repository template for projects using the Sharp Architecture framework.  We created a folder structure and pre-downloaded all binaries and their dependencies so that new repositories would automatically have this structure and these resources.  This same approach could be taken for any number of project types relevant to your organization.

Using these same techniques you could make your template more robust including common resources such as:

• NANT build script templates
• Visual Studio .sln or .csproj files (although your users would most likely have to re-name them for your project)
• Common scripts or tools used by your organization
• Default config files with references to your development or production environment
• etc

For additional information and complete documentation on creating Repository Templates, please see the Sublime documentation.