Sublime Subversion

Let me paint you a picture.  It’s 5 minutes till 5 at the end of a long work week and you’ve just wrapped up a new feature on your current project.  It’s time to commit those changes and head out for the weekend.  You pull up TortoiseSVN, write a helpful and descriptive commit message (feeling vastly superior to your less-than-thorough colleagues with their randomly generated commit messages), and hit the Commit button.

AAAAAAAARRRRRRGGGGHHH!!!!

AHHHH!  The last thing you want to do is spend the next hour picking through other people’s changes to make sure your code still works.

Now here’s a little secret about developing software with a team.  Merging is just a fact of life.  Unless you want to go back to the old lock/modify/unlock model you’re going to have to merge your changes with someone else’s at some point.  But with today’s software, most merges can be handled automatically by our development tools.  The only real problem arises when you have to merge lots of changes all at once across many different parts of your code-base.  Resolving conflicts (deciding what code to keep when two changes contradict each other) requires intimate knowledge of your code and your colleague’s code.  Not the activity for 5PM on a Friday!

There are two keys to keeping merges simple:

1. Commit frequently with simple, focused changes
2. Update your working copy after each of your colleagues commits changes which may impact you

Commit Frequently with Simple, Focused Changes

As you are making changes to your code, keep a mental note when you shift from one focus to another.  Are you fixing a bug?  Developing a new feature?  Optimizing a query?  As soon as you shift from one focus to another – assuming you are finished with the first thing – commit those changes along with a short, simple description of your changes.  This is good practice in general, but it also greatly reduces confusion when a team member has to merge their changes with yours.  A small, focused change is much easier for someone who isn’t familiar with your code to comprehend.  So if I have to merge my changes with yours, and I see that your change fixes a certain bug, I can be sure to pay attention to that detail when merging so as not to re-introduce the bug into the code-base.

Update your Working Copy Frequently

Staying up to date with the activities of your team is a critical step in reducing and even eliminating the pain associated with merging.  If you see that a colleague committed a change to a file you are working on, you can update your working copy right then and there.  This offers two significant advantages: (1) you will be dealing with smaller conflicts that are usually much simpler and (2) the other developer is probably still in the office if you have any questions.

This is where Sublime enters the picture.  Sublime allows you to stay up to date with your development team using email notifications and RSS feeds.  If you enable email notifications on your project repositories you will be notified immediately when another users commits changes.  If you don’t like email, you can use the personalized activity stream RSS feed which notifies you whenever a user commits changes to any of your repositories.  Both of these methods allow you to quickly scan the commit details including the commit message, and changed files so you can determine if their changes might conflict with your work.

Viewing the Commit RSS feed in Outlook

The amazing thing about these two practices is that (a) they are actually pretty simple to implement and (b) the time and mental energy required for merging changes practically vanishes.  And it’s not just that you are spreading that hour merging commits across the day so it doesn’t seem like it takes as much time, it really does take less time.  The reason for this is that you are eliminating the mental ramp-up time required when you save this for the end of the day or worse, the end of the week.

A good way to think about this is efficiency in testing and fixing bugs.  Fixing bugs while your team is actively engaged in development is vastly more efficient than fixing bugs after your team has disbanded and moved on to something else.  While development is in progress, everyone is already engaged and can rapidly jump to the source of a problem, get it fixed, and move on.  Your brain operates the same way.  While you are working on a particular piece of code, your brain keeps a mental model of that code so that you can quickly navigate from one area to another implementing features.  If you have to merge changes and you already have that mental model, you know exactly how those changes will impact your code and what needs to be done to resolve any impacts.

These two practices really aren’t that hard to implement with your team.  Sublime has features that are designed specifically for making this easy, but even if you are using a different product you can still work to implement these practices.  Once you have worked this way for a few days I guarantee you’ll never want to go back.

We just pushed the final bits live and you can download Sublime 1.2 now!  Version 1.2 can be installed right over top of your existing Sublime 1.1 installation so upgrading is a snap!

We are super excited about Sublime 1.2 and feel that it’s the best version of Sublime yet.  Version 1.2 introduces a slew of new features and enhancements.  Check out the release notes for the complete list of new features and bug fixes included.  Below are the highlights:

• Path-based permissions. It is now possible to manage path-based permissions within your repositories.
• Personalized Dashboard. Each user gets their own personalized dashboard showing them their active repositories and activity stream.
• Activity Stream RSS Feed. Keep track of all of your repositories from a single, dynamic RSS feed.
• Active Directory Groups when using LDAP Authentication. Use existing Active Directory groups when managing repository permissions.
• Advanced Repository Templates. Repository templates can contain files and content which will be personalized to the details of your new repository.  Included are two new templates for Visual Studio projects.  Each template will pre-populate the new repository with a Visual Studio solution file based on the name of the repository.

I LOVE this new feature.  I find it to be incredibly useful on a daily basis.  And building tools that you yourself find useful and productive is one of the most rewarding aspects of this business.

Sublime 1.2 ships with a new default home page which is personalized for each user.  It shows a list of active repositories that you have access to, and an activity stream for those repositories.  This makes it very easy to quickly access the repositories you care about, and see exactly who is committing to those repositories – all in one, streamlined view.

New personalized dashboard shows your repositories and latest activity

On the left side of the dashboard is a list of any repository that you have access to, which has been committed to in the last 15 days.  This makes it very easy to quickly access repositories that you have access to and it automatically filters out old or unused repositories.

On the right side of the screen is a list of the most recent commits to these active repositories.  This single, unified view lets you easily see which users are committing to the repositories you care about.

In addition, this activity stream is exposed as an RSS feed.  This lets users subscribe to a single RSS feed for all of their repositories without having to subscribe to each individual repository.

All in all we feel that this is a great feature for developers.  It helps everyone easily stay up to date with the activities of their team.

We’re a bit behind releasing 1.2.  We had hoped to release in July and we may still, but if we don’t make it before the end of this week we’ll definitely have it ready to go by August 6th.  In the meantime, here’s another preview into the new features in 1.2.

Those of you using LDAP / Active Directory for authentication will be pleased to know that with 1.2 it will be possible to use AD groups for permission management.  This is accomplished by creating an “LDAP Group” and entering the name of the group in your LDAP directory.  For example, if you wanted to add “Domain Users”, you would simply create a new LDAP group in Sublime named “Domain Users”.  The group membership will be automatically populated based on the users in the Domain Users active directory group.  Sublime will then keep this membership up to date automatically.  So whenever you add new users to the group in active directory, those additions will be automatically picked up by Sublime.

Sublime is also smart enough to only give access to users who you want using Sublime in the first place.  For example, if you have configured Sublime to exclude all users except those in a given OU, Sublime will respect those rules when pulling in users from your AD groups.  So even if a user exists in that group, if they aren’t in the OU you specified they won’t be granted access to Sublime or Subversion.

That’s it for this post.  Next time we’ll talk about the new user dashboard and user RSS feeds – promise!

As you may know, last week saw the Subversion Vision Conference in New York City. This wasn’t a large event, rather it was a small meeting of several core members of the Subversion team. Their goal wasn’t marketing, community building, or flag waving, but rather to step back a moment and take a look at where Subversion was headed. And although they didn’t come away with a finalized roadmap or vision statement, it does appear that some good progress was made on mapping out the direction for Subversion.

I think this is great and I’m glad to see the Subversion team taking some time to map out the future. But I do wish we had seen some indication that Subversion may evolve to incorporate some distributed functionality. However, according to Michael Pilato the immediate plan appears to be to “Maintain a centralized approach to version control”.

Considering the composition of the team – most members being from companies such as Collabnet or WANdisco who serve the enterprise – this isn’t exactly surprising. Centralization is still a key concern for enterprise IT organizations who need to maintain tight control on their systems for compliance and governance reasons. But I hate to see the team dismiss different approaches to source control simply because it doesn’t appeal to the enterprise yet.

I think distributed version control systems have some great advantages and not just in the open source space. I think cheap, personal branching and merging is an extremely powerful tool. From personal experience I can tell you that it can completely change your development workflow for the better. My opinion is that anything that stops a developer from committing changes is a bad thing. Your version control system is your safety net – it should be there to support you, not to impose speed bumps.

I’ve watched the rise in popularity of DVCSs over the last few years. Git made the DVCS sexy but they’ve been around for quite a while. Microsoft supporting Mercurial on Codeplex is just one example of DVCSs being taken seriously by very large players.

But even though distributed systems are the hot “new” thing, most organizations – big or small – still place a high value on centralization. As a small business using Subversion, I see the value in having one location with all of our valuable IP. I have one asset to back up, one place to go to find repositories, one place to go to manage access, etc. These are all great arguments for a centralized system but I think there can be a middle ground.

The combination of a centralized version control system supporting distributed working copies could be extremely powerful. Allowing your developers to commit, branch, merge, and revert locally and then pushing changes up to the master repository as appropriate seems like the best of both worlds.

This is where the Git users out there yell at me and tell me that Github already works this way. I’m aware of that and have used it myself. It’s a great workflow – but it’s not ready for the enterprise, or even smaller companies. It will work for some, but not for the majority.

This is where I think the Subversion team can make some serious traction – not just supporting the status quo – but really moving the product forward in a new and exciting way. I think that Subversion is in the unique position of being able to take advantage of the benefits that distributed systems offer while at the same time serving those who care about centralization.

In the coming months I expect that the Subversion team will make their roadmap public and I’m excited to see what’s on it. I really like what I’ve been hearing about the Working Copy changes (no more .svn folders littering my working copy!) and love to see the momentum in the community. Subversion is still hands-down the best source control solution for small, medium and enterprise organizations, but it can always be better. I hope that the core team will consider some of these opportunities when mapping out Subversion’s future.

Over the last couple of days we have started posting some short tutorial videos. Initially these videos focus on how to use Sublime but over time we will expand the scope to cover Subversion topics as well.

• Tutorial 1: Creating Subversion Repositories in Sublime
• Tutorial 2: Checking out a Repository

Please let us know if these videos are helpful and if you have any suggestions on features to cover or ways we can improve them.

Even if you are new to Subversion you have likely heard terms like “trunk”, “branching” and “tags” thrown around.  If you are coming from a source control system like CVS you may already be familiar with these terms but if you are coming from a tool like Visual Source safe they may be new.  This post provides an introduction to these concepts in Subversion.  For more in-depth information please see Chapter 4 of the Subversion Book.

First things first – the trunk.  In Subversion, the term trunk refers to your main source code tree.  If you are starting out a new project, you will spend the majority of your time making changes and committing them to the trunk of your repository.  This is because early in a project there are fewer situations where you need branches or tags (there are certainly exceptions depending on your team and development practice, but this is an introductory article and we won’t get into that).

Now, let’s say your project is moving along and getting fairly stable, but you want to experiment with adding a new feature.  You could simply start making changes in your working copy and see how they work out without ever committing them, but this would be a mistake.  If this is a large set of changes your work could span multiple days – even weeks.  At some point your manager is going to wander over to your desk and inform you that there is a critical change that needs to be made RIGHT NOW. Now you’ll need to set aside those experimental changes, get a fresh copy of the trunk, commit those, somehow port those changes into your experimental code, etc etc.  It gets complicated fast.

Branching is specifically designed for situations like this.  A branch is a copy of your source code – typically a copy of the trunk – with a special name like “experimental feature”.  It sets up a space for you to work and make changes without committing to the main trunk.  Once the code in your branch is complete, you can merge your changes back to the trunk.  In addition, as new changes are committed to the main trunk, you can merge those changes into your branch so your branch is up to date with the latest changes from the trunk.  This is a great habit to get into by the way because it makes the process of merging your branch back to the trunk much much easier when the time comes.  Not to mention the fact that you are much more likely to catch potential integration issues early.

I’ll come back to branching in a bit, but now let’s move to tagging.

A tag is simply a named version of your code at a particular point in time.  For example, let’s say you’ve reached version 1.0 of your project.  Rather than having to remember that version 1.0 was revision 638, you can create a tag and give it the name of “1.0″.  Best practice is to create a tag for every major release or milestone (beta, 1.0, 1.1, 2.0, etc) but you can create tags for any point in time you think developers will care about in the future.

So we’ve covered the basic concepts, now I’m going to let you in on a little secret.  Ready?  In Subversion, there is literally nothing special about a branch, or a tag, or the trunk for that matter.  It’s just a directory in your repository which holds a version of your source code.  That’s it.  In most Subversion repositories the directory for storing branches is called, well “branches”, but that is just a naming convention.  You could call yours “superman” and Subversion would still behave exactly the same.

Here’s what I mean.  Most repositories will have the following structure at their root:
/branches
/tags
/trunkEach path above is simply a directory in your repository.  Trunk is self explanatory – it contains your main development tree.  Branches is a folder which will contain sub-folders, one for each branch.  Tags is the same – a folder which will contain sub-folders for each tag.

Here’s what your repository might look like with several tags and branches:

/branches
/branches/new_feature <- branch for implementing a new feature
/branches/1.0_maint <- branch for doing maintenance on version 1.0
/tags
/tags/0.9 <- version 0.9 release
/tags/1.0 <- version 1.0 release
/trunkWhen you create a branch, or a tag, Subversion is simply copying your current code usually from the trunk (although not required) into a new folder in /branches or /tags. Once your code is copied, you can check out a new working copy from that location, make changes, commit, etc. Since your working copy points at the branch folder rather than the trunk, changes you make will only be committed to the branch and not to the trunk. The act of merging is essentially picking the changes you have made and moving them back into the trunk. Subversion gives us tools to make this easier, but it essentially boils down to just that.

Now, why is this important? Well for one it means you aren’t stuck with the standard naming convention of branches, tags, and trunk. If you prefer something like “variations”, “labels”, and “root” it is entirely up to you – Subversion won’t care. However, unless you have a compelling reason to change, it’s best to stick with branches, tags, and trunk because it is widely accepted as the standard naming convention.

Another reason this is important is that Subversion isn’t going to limit you from making mistakes like checking out a tag and making changes to it. Making changes to a tag goes against the entire purpose of a tag because a tag should be a static point in the history of your code – not something you make changes to. If you need to make changes to version 1.0 outside of your trunk (maintenance for example) you should make a new branch using your 1.0 tag as the source (step by step instructions is beyond the scope of this article).

So that’s it. Branches, tags, and trunk are simply standard folders in your repository set up to help you manage your development process. You can change these conventions if you wish, but you should have a solid reason for doing so. You can also add to this structure if you need additional controls for your development team or process.

Before closing I want to make one point about how your code is copied to a branch or a tag. Subversion is smart enough not to make an entire duplicate copy of your code each time you create a branch or tag. It simply creates a pointer to the revision your branch is based on. When you make a commit to the branch, Subversion stores the difference between the modified version and the original file. It does not create a full copy of the file.

Over the coming months we will be releasing the roadmap for Sublime. This will include new features and updates for version 1.0 (which are available free to all current 1.0 users) as well as future plans. To make sure we are planning the right features for Sublime we have set up a user forum to capture your feedback and ideas. Here you can vote on ideas submitted by other users, or submit your own suggestions.

Currently the top contenders are:

• SharePoint integration – creating a repository will create or connect to a SharePoint project site
• Repository History RSS feed – expose an RSS feed on repository commits – may be used as an alternative to email announcements
• Path-based permissions – allow permissions to be set on paths within a repository
• Active Directory Groups – allow AD security groups to be used when assigning permissions

There are plenty of other ideas as well ranging from an enhanced repository browser, incremental backups, and security/permission audit.

Please help us make Sublime the best Subversion server available by providing your input and ideas!

Sublime will automatically send email notifications to your developers after a commit is made.  You can enable email notifications for your repositories by navigating to the Repository in Sublime, clicking the Edit Settings link, and changing the Email Notification setting to “On”.

By default, each email notification looks like this:

Default subversion email notification

However, this can be customized by editing an HTML template file located on your Sublime subversion server.

1. Connect to your server using Remote Desktop
2. Locate the “bin” folder inside your Sublime installation directory (C:\Program Files\Sublime\bin by default).
3. Make a backup copy of the file “EmailTemplate.htm”.  This is optional, but recommended so that you will be able to revert to the default template if you wish.
4. Now, using a text editor such as notepad, open “EmailTemplate.htm”.
5. Modify the HTML to customize the body of the email message.  When Sublime sends an email notification, it replaces certain placeholder tokens in the email template with actual commit details.  For example, “{REPOS}” is replaced with the name of the repository.  A complete list of placeholder tokens is listed below.
6. Save “EmailTemplate.htm”.

Here’s an example of a customized email template.  We have added our company logo at the top, and excluded some of the commit details.

Customized email notification

Placeholder Token Reference

The table below contains the complete list of placeholder tokens which may be used in your email notification template. The example for each is taken from the screen shot below the table so you can see exactly where the token is used in the email notification.

Default subversion email notification

Sublime automatically backs up all of your Subversion repositories on a schedule you define.  In this short article we will walk through the process of configuring your backups, monitoring your server to ensure backups are happening, and restoring a repository.

1. Configuring Backups

To configure your backups, navigate to the Administration section in Sublime and click the “Backups” tab.  First, enter a path for your repositories to be backed up to.  When you install Sublime, it will default this path to be a Backup folder within your Sublime installation directory.  However, it is strongly recommended that you back your repositories up to a network device such as a NAS or remote network share.

Next, enter a time of day and the days of the week that your backups should occur.  Generally this should be a time of the day that users are not using Subversion.  However, Sublime uses the subversion “hotcopy” command when making backups so even if your users are still accessing Subversion you are not at risk of corrupting data or interrupting the backups.

Lastly, enter a backup account to be used when performing the backups.  This account must have read/write access to the Backup path.  It will also be added to the local administrators group on your subversion server.

Scheduling subversion backups in Sublime

2. Monitoring Backups

Each time Sublime backs up your Subversion repositories, it will write an entry to the Windows Event Log.  You can monitor this directly from the Sublime web interface, or by connecting to the server using remote desktop.  Each entry in the windows event log will list the number of repositories backed up, the number of errors (if any), and the duration for the backup.

Successful Subversion backup in the Windows Event Log

If you are using Windows Server 2008, you can create a Custom View to show only Subversion backup messages.  Simply create a new Custom View to display “Information” events filtered where the Source is “Sublime.SubMaint”.

3. Restoring a Subversion Repository

Restoring a repository is essentially copying the backup repository from the backup location back to your Subversion server.  However, as a best practice we recommend the following steps:

1. Take your Subversion service offline.  This ensures that while the restore is underway, your developers cannot commit to the repository.  If you are running Sublime in SVNSERVE mode you can take the Subversion service offline from the administration home page.  If you are using Apache you will have to stop the service manually in the Services console in Windows.
2. Locate the folder containing the repository you want to restore on both your Subversion server, and in the backup location.  The backup location should be a mirror of your repositories folder on your Subversion server, so locating the correct repository should be easy (see below for tips on identifying a repository folder).
3. On your subversion server, delete or rename the folder containing the repository you want to restore.  Then, copy the repository folder from the backup location over to your subversion server.

Let’s look at an example.  In the screen shot below, the left folder is the Repositories folder on our Subversion server.  This contains all of our current “live” repositories.  The folder on the right is our backup location.  If we wanted to restore the “CommonLibs” repository, first we would delete “CommonLibs” from the “Archive” folder on the left.  Then we would copy the “CommonLibs” folder from the backup location on the right and paste it in the “Archive” folder on the left.  The result would be a fully restored version of the “CommonLibs” repository from the time the last backup ran.

The Backup location is a mirror of your Repository location

Once you have restored your repository you can bring the Subversion service back online.  It is also a good idea to reset IIS (open a command prompt and type “iisreset”) to ensure that the Sublime cache is cleared.

TIP: How to locate a repository folder

If you are new to Subversion it may not be clear exactly what constitutes a repository.  In short, a repository consists of a folder containing a number of specific sub-folders and files.  So when we say “repository” we are talking about the parent folder and all child folders and files.

You can identify the repository parent folder by looking at its children.  A repository will contain the following child folders: conf, db, hooks, locks.  It will also contain a file called “format” and there may or may not be a README.txt.  In the following screen shot, we are looking at the content of “CommonLib”  You can see that it contains these folders, thus “CommonLib” is a repository.  The parent folder “Archive” is not – it’s merely a folder.

Identifying a repository folder