Sublime Subversion

Over the last couple of days we have started posting some short tutorial videos. Initially these videos focus on how to use Sublime but over time we will expand the scope to cover Subversion topics as well.

• Tutorial 1: Creating Subversion Repositories in Sublime
• Tutorial 2: Checking out a Repository

Please let us know if these videos are helpful and if you have any suggestions on features to cover or ways we can improve them.

Apologies for the radio silence recently.  While finishing up Sublime 1.1 we decided to completely re-write the Sublime installer.  We had been using an off-the-shelf product for our installer which worked okay initially.  However, after digging into more of the details we decided to switch to the excellent open-source Wix installer.  While we believe this is a good long term decision, it did add quite a bit of development time to 1.1.  We’re working the last bugs out of the installer now and should have 1.1 released soon.

So what’s new in 1.1?  Well the biggest new feature is RSS Feeds.  With Sublime 1.1, each repository will expose an RSS feed of the commit history.  This can be a great alternative to email-based commit notifications.  Each item in the RSS feed will show the time of the commit, the author, the commit message, and the list of changed files.

In addition to repository RSS feeds we added an RSS feed for administrators.  This feed will contain important information about the server such as the success or failure of repository backups, information about new users or repositories created, etc.

We’ve been using both of these features on our internal servers for a couple of months and really like them.  Hopefully you will two.

In addition to those features we’ve included a number of minor enhancements and bugfixes including:

• Turn On/Off Repository folder hierarchy.  This allows administrators to specify whether repositories will be created in a folder hierarchy or a flat list.
• Better support for non-English Subversion servers
• UI bug fixes including better paging support for repositories with a large number of commits
• Email notifications enabled by default for new repositories

Thanks to all of you who have sent us feedback and suggestions.  Once 1.1 is released we’ll post our plans for 1.2.  Remember, 1.x releases of Sublime are free upgrades to anybody who has purchased a Sublime license.

The Board of the Apahce Software Foundation (ASF) has voted to approve the graduation of Subversion from incubation.  Subversion is now a full-fledged ASF project.  This is just one more step in the life of Subversion and one that will hopefully encourage greater community involvement and innovation.

Even if you are new to Subversion you have likely heard terms like “trunk”, “branching” and “tags” thrown around.  If you are coming from a source control system like CVS you may already be familiar with these terms but if you are coming from a tool like Visual Source safe they may be new.  This post provides an introduction to these concepts in Subversion.  For more in-depth information please see Chapter 4 of the Subversion Book.

First things first – the trunk.  In Subversion, the term trunk refers to your main source code tree.  If you are starting out a new project, you will spend the majority of your time making changes and committing them to the trunk of your repository.  This is because early in a project there are fewer situations where you need branches or tags (there are certainly exceptions depending on your team and development practice, but this is an introductory article and we won’t get into that).

Now, let’s say your project is moving along and getting fairly stable, but you want to experiment with adding a new feature.  You could simply start making changes in your working copy and see how they work out without ever committing them, but this would be a mistake.  If this is a large set of changes your work could span multiple days – even weeks.  At some point your manager is going to wander over to your desk and inform you that there is a critical change that needs to be made RIGHT NOW. Now you’ll need to set aside those experimental changes, get a fresh copy of the trunk, commit those, somehow port those changes into your experimental code, etc etc.  It gets complicated fast.

Branching is specifically designed for situations like this.  A branch is a copy of your source code – typically a copy of the trunk – with a special name like “experimental feature”.  It sets up a space for you to work and make changes without committing to the main trunk.  Once the code in your branch is complete, you can merge your changes back to the trunk.  In addition, as new changes are committed to the main trunk, you can merge those changes into your branch so your branch is up to date with the latest changes from the trunk.  This is a great habit to get into by the way because it makes the process of merging your branch back to the trunk much much easier when the time comes.  Not to mention the fact that you are much more likely to catch potential integration issues early.

I’ll come back to branching in a bit, but now let’s move to tagging.

A tag is simply a named version of your code at a particular point in time.  For example, let’s say you’ve reached version 1.0 of your project.  Rather than having to remember that version 1.0 was revision 638, you can create a tag and give it the name of “1.0″.  Best practice is to create a tag for every major release or milestone (beta, 1.0, 1.1, 2.0, etc) but you can create tags for any point in time you think developers will care about in the future.

So we’ve covered the basic concepts, now I’m going to let you in on a little secret.  Ready?  In Subversion, there is literally nothing special about a branch, or a tag, or the trunk for that matter.  It’s just a directory in your repository which holds a version of your source code.  That’s it.  In most Subversion repositories the directory for storing branches is called, well “branches”, but that is just a naming convention.  You could call yours “superman” and Subversion would still behave exactly the same.

Here’s what I mean.  Most repositories will have the following structure at their root:
/branches
/tags
/trunkEach path above is simply a directory in your repository.  Trunk is self explanatory – it contains your main development tree.  Branches is a folder which will contain sub-folders, one for each branch.  Tags is the same – a folder which will contain sub-folders for each tag.

Here’s what your repository might look like with several tags and branches:

/branches
/branches/new_feature <- branch for implementing a new feature
/branches/1.0_maint <- branch for doing maintenance on version 1.0
/tags
/tags/0.9 <- version 0.9 release
/tags/1.0 <- version 1.0 release
/trunkWhen you create a branch, or a tag, Subversion is simply copying your current code usually from the trunk (although not required) into a new folder in /branches or /tags. Once your code is copied, you can check out a new working copy from that location, make changes, commit, etc. Since your working copy points at the branch folder rather than the trunk, changes you make will only be committed to the branch and not to the trunk. The act of merging is essentially picking the changes you have made and moving them back into the trunk. Subversion gives us tools to make this easier, but it essentially boils down to just that.

Now, why is this important? Well for one it means you aren’t stuck with the standard naming convention of branches, tags, and trunk. If you prefer something like “variations”, “labels”, and “root” it is entirely up to you – Subversion won’t care. However, unless you have a compelling reason to change, it’s best to stick with branches, tags, and trunk because it is widely accepted as the standard naming convention.

Another reason this is important is that Subversion isn’t going to limit you from making mistakes like checking out a tag and making changes to it. Making changes to a tag goes against the entire purpose of a tag because a tag should be a static point in the history of your code – not something you make changes to. If you need to make changes to version 1.0 outside of your trunk (maintenance for example) you should make a new branch using your 1.0 tag as the source (step by step instructions is beyond the scope of this article).

So that’s it. Branches, tags, and trunk are simply standard folders in your repository set up to help you manage your development process. You can change these conventions if you wish, but you should have a solid reason for doing so. You can also add to this structure if you need additional controls for your development team or process.

Before closing I want to make one point about how your code is copied to a branch or a tag. Subversion is smart enough not to make an entire duplicate copy of your code each time you create a branch or tag. It simply creates a pointer to the revision your branch is based on. When you make a commit to the branch, Subversion stores the difference between the modified version and the original file. It does not create a full copy of the file.

Over the coming months we will be releasing the roadmap for Sublime. This will include new features and updates for version 1.0 (which are available free to all current 1.0 users) as well as future plans. To make sure we are planning the right features for Sublime we have set up a user forum to capture your feedback and ideas. Here you can vote on ideas submitted by other users, or submit your own suggestions.

Currently the top contenders are:

• SharePoint integration – creating a repository will create or connect to a SharePoint project site
• Repository History RSS feed – expose an RSS feed on repository commits – may be used as an alternative to email announcements
• Path-based permissions – allow permissions to be set on paths within a repository
• Active Directory Groups – allow AD security groups to be used when assigning permissions

There are plenty of other ideas as well ranging from an enhanced repository browser, incremental backups, and security/permission audit.

Please help us make Sublime the best Subversion server available by providing your input and ideas!

Sublime will automatically send email notifications to your developers after a commit is made.  You can enable email notifications for your repositories by navigating to the Repository in Sublime, clicking the Edit Settings link, and changing the Email Notification setting to “On”.

By default, each email notification looks like this:

Default subversion email notification

However, this can be customized by editing an HTML template file located on your Sublime subversion server.

1. Connect to your server using Remote Desktop
2. Locate the “bin” folder inside your Sublime installation directory (C:\Program Files\Sublime\bin by default).
3. Make a backup copy of the file “EmailTemplate.htm”.  This is optional, but recommended so that you will be able to revert to the default template if you wish.
4. Now, using a text editor such as notepad, open “EmailTemplate.htm”.
5. Modify the HTML to customize the body of the email message.  When Sublime sends an email notification, it replaces certain placeholder tokens in the email template with actual commit details.  For example, “{REPOS}” is replaced with the name of the repository.  A complete list of placeholder tokens is listed below.
6. Save “EmailTemplate.htm”.

Here’s an example of a customized email template.  We have added our company logo at the top, and excluded some of the commit details.

Customized email notification

Placeholder Token Reference

The table below contains the complete list of placeholder tokens which may be used in your email notification template. The example for each is taken from the screen shot below the table so you can see exactly where the token is used in the email notification.

Default subversion email notification

I came across this article today in another forum.  It’s a translation of a Russian article about how the source code for websites that use Subversion can be obtained if they don’t remove or block the hidden .svn folders from their site structure.

Basically, since Subversion maintains a text-based copy of your files in the hidden .svn directory, it’s possible to view the source directly if you know that a given website is using Subversion.

For example, if you want to view the source to index.php on a given site, you could try a URL like this:
mysite.com/.svn/text-base/index.php.svn-base
The server would see that as a request for a .svn-base file, not a .php file and it would simply serve up the source rather than running through PHP.

There are two ways to protect you against this.  First, as I mentioned in a previous post, DON’T DEPLOY .svn FOLDERS.  The svn export command lets you grab a full tree of your source code without the .svn folders – so use that if you are deploying a website.

Second, you can set up your web server to block .svn folders.  The article has some examples of how to do that for apache and nginx.  I haven’t tried this on IIS yet but I’ll give it a shot and update this post with steps to protect yourself on Windows.

Sublime automatically backs up all of your Subversion repositories on a schedule you define.  In this short article we will walk through the process of configuring your backups, monitoring your server to ensure backups are happening, and restoring a repository.

1. Configuring Backups

To configure your backups, navigate to the Administration section in Sublime and click the “Backups” tab.  First, enter a path for your repositories to be backed up to.  When you install Sublime, it will default this path to be a Backup folder within your Sublime installation directory.  However, it is strongly recommended that you back your repositories up to a network device such as a NAS or remote network share.

Next, enter a time of day and the days of the week that your backups should occur.  Generally this should be a time of the day that users are not using Subversion.  However, Sublime uses the subversion “hotcopy” command when making backups so even if your users are still accessing Subversion you are not at risk of corrupting data or interrupting the backups.

Lastly, enter a backup account to be used when performing the backups.  This account must have read/write access to the Backup path.  It will also be added to the local administrators group on your subversion server.

Scheduling subversion backups in Sublime

2. Monitoring Backups

Each time Sublime backs up your Subversion repositories, it will write an entry to the Windows Event Log.  You can monitor this directly from the Sublime web interface, or by connecting to the server using remote desktop.  Each entry in the windows event log will list the number of repositories backed up, the number of errors (if any), and the duration for the backup.

Successful Subversion backup in the Windows Event Log

If you are using Windows Server 2008, you can create a Custom View to show only Subversion backup messages.  Simply create a new Custom View to display “Information” events filtered where the Source is “Sublime.SubMaint”.

3. Restoring a Subversion Repository

Restoring a repository is essentially copying the backup repository from the backup location back to your Subversion server.  However, as a best practice we recommend the following steps:

1. Take your Subversion service offline.  This ensures that while the restore is underway, your developers cannot commit to the repository.  If you are running Sublime in SVNSERVE mode you can take the Subversion service offline from the administration home page.  If you are using Apache you will have to stop the service manually in the Services console in Windows.
2. Locate the folder containing the repository you want to restore on both your Subversion server, and in the backup location.  The backup location should be a mirror of your repositories folder on your Subversion server, so locating the correct repository should be easy (see below for tips on identifying a repository folder).
3. On your subversion server, delete or rename the folder containing the repository you want to restore.  Then, copy the repository folder from the backup location over to your subversion server.

Let’s look at an example.  In the screen shot below, the left folder is the Repositories folder on our Subversion server.  This contains all of our current “live” repositories.  The folder on the right is our backup location.  If we wanted to restore the “CommonLibs” repository, first we would delete “CommonLibs” from the “Archive” folder on the left.  Then we would copy the “CommonLibs” folder from the backup location on the right and paste it in the “Archive” folder on the left.  The result would be a fully restored version of the “CommonLibs” repository from the time the last backup ran.

The Backup location is a mirror of your Repository location

Once you have restored your repository you can bring the Subversion service back online.  It is also a good idea to reset IIS (open a command prompt and type “iisreset”) to ensure that the Sublime cache is cleared.

TIP: How to locate a repository folder

If you are new to Subversion it may not be clear exactly what constitutes a repository.  In short, a repository consists of a folder containing a number of specific sub-folders and files.  So when we say “repository” we are talking about the parent folder and all child folders and files.

You can identify the repository parent folder by looking at its children.  A repository will contain the following child folders: conf, db, hooks, locks.  It will also contain a file called “format” and there may or may not be a README.txt.  In the following screen shot, we are looking at the content of “CommonLib”  You can see that it contains these folders, thus “CommonLib” is a repository.  The parent folder “Archive” is not – it’s merely a folder.

Identifying a repository folder

Repository Templates are a great way to reduce the time required for setting up a new project, and encourage consistency across projects.  Consistency is important especially in an organization with many different repositories spread over different projects and development teams.  Having a consistent repository structure drastically reduces common errors made by new developers or even experienced developers in an unfamiliar repository.

Setting up Repository Templates in Sublime is as easy as creating a folder structure on your Sublime server containing the files and folders you want to include in your new project.  In this walk-through we’re going to create a Repository Template for new projects using Sharp Architecture.  Sharp Architecture is a perfect candidate for a Repository Template because there are quite a few dependencies that can take a while to download and organize.

In the following steps we’ll be creating a Repository Template for Sharp Architecture, but the same steps can be applied to any number of project types you may have.

1. Create the new Repository Template Folder

The first step is to create the new Repository Template folder on your Sublime/Subversion server.

1. Connect to your server using Remote Desktop
2. Navigate to the “Templates” folder beneath your Sublime install location (typically C:\Program Files\Sublime\Templates)
3. Create a new folder inside Templates called “SharpArch” (note: the folder name can be different from what your users see when choosing a template)
   

   Create a new folder for the template

4. Open the new folder and create the standard Subversion trunk, branches, and tags folders.
   

   Create the branches, tags, and trunk folder

5. Open the “trunk” folder you just created.  Now it’s time to start creating the folder structure for your new project.  For starters, Sharp Architecture projects tend to have a defined top-level folder structures for the different elements of a project: app, build, db, tests, etc.  So let’s create those folders now.
   

   Create the project folder structure

6. Next, if you like you can create README.txt files in each folder explaining what should be placed there.  This is optional of course but can be useful for new developers who may not be familiar with the project structure.

2. Download Sharp Architecture binaries

Now that our basic structure has been created, it’s time to download the related Sharp Architecture binaries as well as their dependencies.

1. Open a web browser and navigate to the Sharp Architecture download page.
2. Download and extract the latest revision (or which ever revision you prefer) of the compiled binaries
3. In the extracted folder, locate all of the DLLs and associated files (located within the “bin” folder as of this writing) and copy them all to the “lib” folder we created back in step 5 of the previous section.
   

   Copy Sharp Architecture binaries to template

3. Register the Template with Sublime

The last step is to register the newly created Repository Template with Sublime.

1. Navigate back to the Templates folder and open “templates.txt” in Notepad
2. This file follows a simple INI file format and you should have two templates already listed (Standard and Empty).  Create a new section with the following structure:
   [SharpArch]
name = Sharp Architecture
description = Use for Sharp Architecture based projects.
   

   Insert new template description into templates.txt

3. In the example above, the section header “[SharpArch]” must be the same as the folder name for the template.  But the value for the “name” attribute is what will be displayed to your users.  There is also an optional “default = true” attribute you can add if you want this template to be automatically selected as the default choice when creating new repositories.
4. Save templates.txt and close Notepad.
5. Next you must restart IIS by opening a command prompt and typing
   C:\> iisreset

4. Test the Template

Open a web browser and navigate to Sublime.  Click on the Create New Repository tab and in the Template section, you should see the new “Sharp Architecture” template.

Preview the new template

Wrap-up

In this walkthrough we’ve created a new Repository template for projects using the Sharp Architecture framework.  We created a folder structure and pre-downloaded all binaries and their dependencies so that new repositories would automatically have this structure and these resources.  This same approach could be taken for any number of project types relevant to your organization.

Using these same techniques you could make your template more robust including common resources such as:

• NANT build script templates
• Visual Studio .sln or .csproj files (although your users would most likely have to re-name them for your project)
• Common scripts or tools used by your organization
• Default config files with references to your development or production environment
• etc

For additional information and complete documentation on creating Repository Templates, please see the Sublime documentation.

This article summarizes a few tips and tricks for working with Subversion from the Visual Studio IDE.  Note that these tips assume that you are using TortoiseSVN as your SVN client.  However, they should hold up whether you are using an SVN client integrated with Visual Studio or simply the command line.

1. Ignore bin and obj folders

One of the first things you should do with any new solution or project is to ignore the bin and obj folders that are automatically created when you compile your project.  Here are the steps I follow when adding a new project to your solution:

1. Add the project in Visual Studio
2. Compile once
3. Close Visual Studio and add the new project folder with TortoiseSVN
4. When adding the project folder, be careful not to also add the “bin” and “obj” folders
5. Commit your changes from TortoiseSVN.  When the commit dialog comes up, right-click on the “bin” and “obj” folders and select “ignore”
6. Finish the commit.

2. Ignore .suo and .user files

These files pop up from time to time and you want to watch out for them.  These files are specific to each user, so if they get committed, everyone will be continuously prompted to resolve conflicts because their version differs from the version in Subversion.

3. Close Visual Studio between commits

I’m going to catch a lot of flack for this one because it seems like a pain.  But it’s really not that hard and it will help you avoid making multiple commits for a single change because .csproj or .sln files hadn’t been saved yet.  For example, let’s say you add a new .cs file to your project.  You code it up and are ready to commit.  Switching over to TortoiseSVN you see your new .cs file needs to be committed, but nothing else.  You commit that file and keep working.  However, the .csproj file has been changed to, but it has only been changed in memory and not saved to disk so TortoiseSVN doesn’t see the change.  You are now left in a position where only the new .cs file has been committed, and the .csproj file hasn’t.  Ideally both files would be committed at the same time because together they represent the addition of your new .cs file.

My workflow is this:

1. Decide what feature or bugfix I’m going to work on
2. Open visual studio.
3. Make the changes necessary for that feature/fix only.
4. Test
5. Close visual studio
6. Commit from the root of my working directory to get all files changed
7. Go to step 1

4) Don’t accidentally deploy .svn folders

This one is easy to overlook especially when it comes to ASP.NET projects.  Subversion creates a hidden .svn folder inside each folder of your working directory.  If your deployment “process” consists of copying your ASP.NET project folder and all sub folders (images, css, etc) and pasting them on your web server, you’re going to be copying all of those .svn folders along with it.

There are really two options to avoid this.  The first is to use the “export” command from Subversion.  If you are comfortable installing the Subversion command line client on your target server, then you can run a command like this to deploy your changes:

C:\> svn export svn://myserver/myrepo/trunk C:\inetpub\wwwroot\mywebsite

Once you have done that you’ll also need to copy over the DLLs because the bin folder won’t be included in the repository (assuming you followed tip #1).

A better solution is to use some sort of continuous integration or automated build process.  Cruise Control.NET is a great solution for this sort of thing.

That’s it for now.  I’ll return to this post and update it from time to time as I identify more tips and workarounds.  Please share your own tips by commenting below.