In 1999, I used a Rapid Software Testing approach to identify problems in Visual SourceSafe. I found it was a dangerous, defective product that put the data of its users at risk.
Introduction
In this document, I describe a number of the issues related to Visual SourceSafe (VSS). These observations were based on experiences in 1999 while I was developing a tool to migrate Visual SourceSafe database contents to a competing software configuration management program.
In order to develop this tool, I had to test for problems in extracting data from Visual SourceSafe. Part of the motivation was to find workarounds for the problems, so that the migration tool would be able to handle them; another part of the motivation was to find things that simply would never work, so that the tool would know when to give up.
This was a case of Rapid Software Testing that took place more or less at the same time that James Bach was beginning to define and use the term.
Testing is an investigative process. Rapid Software Testing is about identifying important information—especially about bugs—about a program when you’re working under extreme constraints of time, resources, and reference material.
Traditional forms of testing insist that you have to have some sort of specification to test effectively. We disagree; testers can find all sorts of information, often not known to those who have some experience with the program, by applying investigative skills and heuristics.
I didn’t have much time to test SourceSafe, and needless to say I did not have a functional specification to work from. I did have SourceSafe’s end-user documentation, and a couple of supplementary papers from Microsoft. I had a good deal of experience with configuration management software. At that point, I was also an experienced and thorough tester.
For oracles, I had Visual SourceSafe’s GUI and command-line front ends (when testing the API); its API (when testing the front ends); my code — the migration tool that I was developing; and a third-party program that I could use, in some circumstances, for comparison with VSS’s behaviour. So even though I had no functional specification, I think you’ll agree, based on the report below, that I was able to test quite effectively.
The number of hits on this page and the postive feedback that I’ve received over the years suggest that other people have found the information to be useful. I attempted to contact a Visual SourceSafe program manager at Microsoft on the problems noted here. The program manager responded to my initial inquiries, but did not reply to a email message that contained most of this document.
So, in this report, I outline some of the errors and problems that I have encountered while attempting to retrieve projects from SourceSafe using its command line, its graphical user interface, and its Automation API. I point out discrepancies and deficiencies in the SourceSafe documentation. I conclude by stating that SourceSafe contains problems that are so fundamental to the product and so easy to reproduce that all users of SourceSafe face substantial risks, and that many will lose access to historical information, which in turn will prevent rebuilding software from source.
I performed my testing using Microsoft Visual SourceSafe Version 6.0. I used the default version 5.0 database format. I eventually applied a patch, bringing SourceSafe to Version 6.0 Service Release 3. I performed my testing again and observed no change in the results described below.
Disclaimer: I have observed and reproduced repeatedly the problems listed below. I cannot guarantee that these problems are reproducible on other systems and other platforms. However, there is no reason for me to suspect otherwise. I urge the concerned reader to attempt to reproduce these problems using simple test projects.
Data Integrity Issues
SourceSafe seems to work reasonably when one uses a simple cycle of creating and then developing source files, checking the files in at given milestones, labeling the project, and so on. However, many simple actions cause the software to fail in various subtle (and sometimes unsubtle) ways.
• A file that has been deleted may easily be purged from the database, preventing previous versions of the project from being rebuilt.
Create a project called A. To this project, add a file named 1.c. Optionally revise the file; then delete it from A. Add a different file called 1.c to A; you will be offered the choice to recover the file, or to purge it from the database. Choose not to recover the deleted file, but instead to add your new file. Display the history of the project, and attempt to Get a version of that project from before 1.c was deleted. You will find that the original version of 1.c is not available, and thus your project cannot be rebuilt.
The user must have destroy rights in order to purge (that is, to permanently delete the file). However, a reliable version control system ought to be able to preserve and retrieve all the historical versions of a file, even if some of those versions were deleted.
• A file that has been renamed will not be retrieved properly from the database for versions before the rename.
Create a project called A. To this project, add a file named 1.c. Revise the file and check it in. Rename the file to 2.c using the SourceSafe Explorer. Display the history of the project. Note that the version before the rename refers to a file called 2.c-the new filename. Get this version of the project to a local directory. You will receive the appropriate, old file-1.c-under the appropriate, old filename.
Now switch to the history of the file, rather than the history of the project. Note that nothing in the file’s history suggests that the file has been renamed. Get the checked-in version of the file-and recall that the checkin happened before the rename. You will find that the file is copied to the new name, not the old one; yet the file was checked in under the old name.
• A file that has been shared into another project, and then deleted from the original project, loses its history.
Create a file in the root project. Check the file out, make some modifications to it, and check it in. Repeat this process a couple of times. Inspect the file’s history. Create a new project called A. Share the file into A, then delete the file from the root project. Open the A project to reveal the file. Highlight the file and attempt to display its history. You will see that there are no items to display; the file’s history has been lost.
• A subproject that has been renamed is improperly tracked in the SourceSafe history, and SourceSafe may retrieve the files within to an inappropriate directory.
Create a project called Foo1. Add one file-1.c-to it. Rename Foo1 to Foo2 using the SourceSafe GUI. Inspect the history for the project Foo2 (choose Recursive, Include File Histories, and Labels). You will see no notification in Foo2’s history of the project name change. If you click on the “Created” item, you will also see that the first version of the project was Foo2, version 1, which is quite incorrect; the project’s name was Foo1 at the time. An error such as this could easily break the batch file or makefile that was used to build earlier versions of a renamed project.
Use the SourceSafe GUI to inspect the history of the single file, 1.c, created in the previous exercise. In the history dialog, highlight the Created action, and select Details. In the File: line at the top of the dialog, you will see that, according to SourceSafe, the file was added to project Foo2, which is incorrect.
Inspect the history of the parent of Foo2. By reading the parent’s history backwards, you will note that first Foo1 was created; again that 1.c was added to Foo2 (still incorrect; Foo2 didn’t exist at this point); and that Foo1 was then renamed to Foo2.
Now, still viewing the history of Foo2’s parent, attempt to Get the version of 1.C that was added by getting the project to which the version applies. You will not obtain the added file in Foo1, where it belongs, but in Foo2.
• The data within the SourceSafe database is open to overwriting or deletion by all of SourceSafe’s users.
SourceSafe requires all users to have unobstructed read/write access to the directories in which SourceSafe’s data is stored. This makes SourceSafe’s database vulnerable to incompetent or malicious users.
• The Destroy command may destroy more than the user might reasonably expect.
A user could reasonably expect that the Destroy command deletes the current working version of a file from the user’s hard drive. A user might also reasonably expect that the Destroy command deletes the file from the current version of the project. The user might not expect that the file is deleted from all previous versions of the project. Thus, one cannot reproduce any build that depends on a destroyed file. Under Automation, even the name of the destroyed file is unavailable. It is of the essence of any configuration management system to be able to reproduce all previous versions of the software.
• The history of shared projects gets events in the wrong order.
Share a project B that contains some files into project A. As you walk through the version history of Project A, you get a Share action for each of the files in Project B. Then and only then do you get notification that Project B was added to Project A.
• The SSARC.EXE archive utility easily generates archives that cannot be successfully restored.
If one exports a project other than the root project using the Visual SourceSafe archive utility SSARC.EXE, objects shared from other projects are inaccessible.
• SourceSafe sometimes locks and unlocks files inconsistently.
I found this report on Usenet (comp.software.config- mgmt) that suggested a problem with VSS and InterDev integration:
Eric Gorely[EGOR@ECOLLEGE.COM wrote in messageUNWGKVVW$GA.275@CPMSNBBSA04] …
Is anyone here using Visual Studio 6.0 and InterDev/VSS integration? I was wondering if you have been having problems with it. We have been having several strange problems that we can’t figure out. Sometimes, even after a project refresh, files aren’t displayed as being locked in InterDev when they really are. Also, when getting latest version, it doesn’t always get the latest version, rather the initial version. We have to check out the file in order to get the latest version. This happens regardless of whether we have a local copy or not. There are other issues as well, but I don’t want to write a novel here!
I have observed the locking problem myself in VSS’s integration with Visual Basic 6.0. That problem was reduced with an update to VSS 6.0 SP 3.
Many problems in SourceSafe are described as “database corruption” in the Microsoft Developer Network Library. However, these are not all cases of corruption (which I would define as a good database gone bad); instead, my belief is that SourceSafe fails to track the data properly from the outset.
Irritants and Inconsistencies
• When you display a project’s history to the screen using the command line SS History command, or pipe the project’s history to a file , the paths to the subprojects are not displayed. The user must figure out the fully-qualified path of the subproject from the report. Interestingly, this restriction does not apply under the GUI or the API.
• VSS’s Share function does not permit a comment.
• VSS’s Rename function does not permit a comment.
• Projects are identified as files in the GUI.
Run a history for any project (the root project will do fine). From the History of Project $/ window, choose any event associated with a project (that is, any item that begins with a $). Choose “Details”. Note that the first line of the History Details window displays File:, followed by the name of the project. Projects are not files; if VSS is displaying details on a project item, it should say “Project”.
• Moves of files and projects are handled inconsistently.
For Project moves, you get a Moved From action in one project and a Moved To action in the other project. When a file has been moved from one project to another, the action that you receive is not “Moved” at all. Instead, you receive the action that a file was shared in one project and then deleted in the first project.
Automation API Issues
• There is no automation call for a complete, accurate version of the project’s history.
Given a particular SourceSafe project, one cannot obtain a clear history for it easily. For each version of the project, one can determine that an item has been added, deleted, or destroyed, but one cannot determine the identity of the item that was added, deleted, or destroyed using any direct method or property within the API.
• Although the Automation API tells you that a file or a subproject was added to a project, the API does not tell you the identity of the added item.
The only way to do it is to determine which file was added is to create two lists; one of the files in the current project, and one of the files in the previous project (in VB, I used a Collection of VSSItems, but you can use whatever data types you find most useful). Compare the two lists, remove the items that are common to both, and you’ve got the item that was added. Note that if there is a problem retrieving the previous version of the project, this algorithm will be unsuccessful.
• Although the Automation API tells you that a file or a subproject was deleted from a project, the API does not tell you the identity of the deleted item.
Determining the identity of a deleted item is a little different from figuring out an added item. A deleted item still exists in the project. The item has a “deleted” property whose value is set to True, but the item is still there; thus a simple comparison of the two projects will reveal no differences. Instead, compare the deleted items in the previous project with the deleted items in the current project. There will be one extra deleted item in the current project. As above, if there is a problem retrieving either the current or previous versions of the project, this algorithm will fail.
• Although the Automation API tells you that a file or a subproject was destroyed in a project, the API does not tell you the identity of the destroyed item.
The algorithm for determining a destroyed item is still different from the previous two. To find a destroyed item, one must enumerate all of the items in the previous version of the project, then enumerate the items in the current version of the project. The previous version will contain one extra item, which was the destroyed item. Note that in most circumstances, an item destroyed in the current project version will have existed as a regular (non-deleted) item in the previous project version, and therefore will move from regular status to destroyed status. However, an item that is marked deleted in our project of interest might be shared into another outside project; in this second project, our item is not deleted. If our item is then destroyed in the second, outside project, things will look odd in our current project; the item will move from deleted status to destroyed status, rather than from regular status to destroyed status.
• The Versions collection does not implement “item” or “count”.
The Versions collection is “odd”, to use the word in Microsoft’s own documentation [Felder, 1995]. Worse than odd, it’s not a proper collection at all. It implements neither the Item nor Count nor Add nor Remove properties (a collection really must implement the first two at least). This irritant has lingered in the product for four years.
• The Versions collection is stored from back to front-and requires an extra step to find the front properly.
When one iterates through the Versions “collection”, the first item is the most recent, and the last is the original version. Since the collection fails to implement the “Count” property, there is no way to get the index number for the first version without iterating through the entire collection.
• Specific VSSItems cannot be linked to their own Version properties; it can only work the opposite way.
For a VSSItem that represents a given revision of a file or project, there is no method for obtaining version information specific to that version of that item—the comment or action that gave rise to it, for instance. You have to do it backwards; you must obtain the item, obtain its Versions collection; iterate through that collection (as above, you must do this backwards, because Versions are stored most-recent first), and get the associated data for each version, and (if necessary) the VSSItem revision to which that version refers.
• The Versions collection contains items other than the VSSItem from which it is derived.
The Versions collection doesn’t always return a version of the item you started with. Create a file, and label the project in which it resides. Iterate through the Versions collection on the file. Note that some of the VSSItems returned to you in the Versions collection apply to the file, and at least one (the label) will apply to the item’s parent project, rather than the VSSItem that you intended to inspect. Thus if you Get the VSSItem to which a version refers, you may get the file of interest or you may get its parent project. I couldn’t believe this when I saw it, but there it is in [Winter, 1995].
• Deleted VSSItems forget their version histories.
In SourceSafe, a deleted file is not really deleted until it is destroyed. The deleted file remains in the SourceSafe database, with its deleted property set to TRUE. The deleted file’s Version collection cannot be set to an IVSSVersions variable; Error -2147221504 is returned instead.
• The version number of a renamed file’s parent cannot be obtained properly.
Create a folder named Foo1. Add a file named Bar. Rename the folder to Foo2. Assign Bar’s Parent property to a new VSSItem named BarParent. If you’re like me, you’ll see that BarParent’s VersionNumber property returns 77737072, instead of 2 as it ought.
• The RECURSYES flag, when used on the Versions collection, doesn’t seem to have any effect.
• Project Names are returned inconsistently.
If you attempt to obtain the Name property of the root project ($/), you get a zero-length string. If you attempt to obtain the Name property of a subproject (say, $/Foo), you get $/Foo. This is inconsistent. Either you should get Foo for the subproject Name, or $ for the project name.
Documentation Issues
• VSSVersion’s property “Action” has a Pinned action that is completely undocumented.
• VSSVersion’s property “Action” has a Purged action that is completely undocumented.
• VSSVersion’s property “Action” has a Recovered action that is completely undocumented.
• VSSVersion’s property “Action” has a Rollback action that is completely undocumented.
• VSSVersion’s property “Action” has an Unpinned action that is completely undocumented.
• Although “Moved” is documented as a VSSVersion action, there is no way to cause such an action to occur with files using the GUI. Any attempt to move a file results in two events; a Share into the destination project, and a delete in the source project.
• The default flags for various automation functions are not documented in [Winter].
One can’t find the default flags for the VSSITem.Get method, for instance. Only in [Felder] can you discover which flags are valid for each method.
• [Winter] describes reserved flags called “USERRONO” and “USERROYES”.
These allegedly serve no purpose in Automation, but “may in the future”. There is no mention of the “USERONO” or “USEROYES” (note the missing R) which are documented in the earlier version of the documentation [Felder]. One gets the feeling they’re making this up as they go along, and that nobody is checking these documents.
• An organizational issue: the methods of VSSItem are listed in a completely random order in [Felder].
• When you look up “Automation” in the online help for VSS, you get a page that says “there is no Help for this topic.”
• Tom Christian’s document, “HOWTO: Write Automation for Visual SourceSafe 5.0/6.0 (ID: Q201431)” [MSDN] contains the following fascinating tidbit: “This article provides sample C++ code for OLE calls. It also describes how to check whether version 5.0 or 6.0 of Visual SourceSafe is running on a computer because OLE Automation written for one version may not run under the other.” All documents here should cross-reference each other.
• The standard APIs, which are disclosed and supported are inadequate for the task of obtaining an accurate history and a robust and accurate migration. However, developers that are planning to release a product to the public may obtain the SCC API from Microsoft. This API is subject to non-disclosure. It is unsupported, and according to Microsoft representatives, will be withdrawn at some future date, which means (one would presume) that this would break the released product. So developers are given the Hobson’s choice of a disclosed, supported API that doesn’t work, or an undisclosed, unsupported API that may work at present, but will be broken in the future.
• In the SourceSafe documentation, the command- line command to add all files and subdirectories to the current project is incorrect.
The command is described as follows:
Adds all files in current folder and all subfolders to the current project:
ss Add -R *
This is incorrect. The command, as listed, adds only the contents of the current directory. Through experimentation (no hints from the documentation), I found that the following command works in accordance with the description above:
ss Add . -r *
References
[Felder]
Felder, Ken. “Visual SourceSafe OLE Automation”, in MSDN Library, October 1999. Redmond, WA: Microsoft, 1995.
[Winter]
Winter, Tim. “Visual SourceSafe 6.0 OLE Automation”, in MSDN Library, October 1999. Redmond, WA: Microsoft, 1999
[MSDN]
Microsoft Developer Network Library, October 1999. Redmond, WA: Microsoft, 1999
Further Links
My correspondent Adam Cogan would like to point people to his list of VSS suggestions at http://www.ssw.com.au/SSW/Standards/BetterSoftwareSuggestions/SourceSafe.aspx.
Another correspondent, Ricardo Drizin, would like me to link to his company’s report on VSS. The company apparently markets a tool for VSS migration; this is intended neither to endorse not to disparage that product.
This essay is copyright ©2003 Michael Bolton. If you liked it, please let me know. If you didn’t, please let me know how I might improve upon it. If you’d like to be notified when I’ve posted another essay that you might enjoy, please click here to be put on my mailing list. If your reply-to address isn’t altered to prevent spam, you don’t have to put anything in the body of the message.
You are welcome to reprint this article for your own use or for your company’s use, as long as you follow the instructions here (https://www.developsense.com/reprints).
Best of all, if you (or your company, or your manager, or your employee) need counselling or instruction in this area, I can help with engaging and informative courses on quality assurance and software testing in plain English that can save your company lots of time and money. Contact me for details. Thanks!