Deployment options?
Tom asked,
Ray, one topic that I'd love to hear about from the audience on your site is the techniques that are used to send files live on a production environment. Do people use a program like xcopy or Robocopy, or do they just push them manually with FTP? What kinds of testing and versioning are commonly used before a file is sent live?
To be honest I suck at this. Well, I don't know if "suck" is the right term, but I don't do anything fancy. I just use FTP. Typically, once a site has gone live, my only changes are new features and bug reports. (Sorry, "Issue Management.") Since this typically involves one or two files, I just FTP them up to live. This has only been a problem once - with a client that had two boxes, and they had replication set up so even if I did forget to move a file, it was fixed within the hour.
You mentioned versioning as well, and yes, I do use versioning (mostly Subversion now), but it isn't tied yet to my deployment process. (If that makes sense.)
So this is probably not the best answer - but as Tom said, he was looking for a range of answers from my readers. (As a side note - how do folks feel about "discussion" topics like this? I almost didn't blog this as I didn't have an 'authoritative' type answer, but he did request feedback from my 'audience' so I figured it was ok. This blog is certainly more than just me, and I love that, and I think my readers do as well. Anyway, if folks have a problem with entries like this, please ping me via email.)
Comments
For single server setups with single directory configuration, using standard remote server in Dreamweaver will work fine in most cases. I often work with a dev directory and once the new features are working fine, I usually diff the changes with something like Visual Slick Edit and only copy what I need to.
And there's always the CF editor a lot of us cut our teeth on, CFStudio/Homesite+, which still supports scriptable deployment and has a wizard that automatically builds the scripts for you.
Anyone with experience in CF .car/.ear/.war deployment care to comment?
----
As a one man shop, I primarily use FTP. Whenever possible, I use secure ftp. I like <a href="http://www.winscp.com/">WinScp (see winscp.com)</a>.
My sop used to be to encrypt (apparently cf7 can bytecode) the source until final payment was received. I became lazy and simply trust my clients now. Not protecting source in someway has cost me 2 or 3 times.
1. Make a nightly process on the deployment server (updates.mycompany.com) that does:
a. MD5/SHA your production source tree. (md5deep is great for this, but it can also be done in CF)
b. Add a database entry for each file that has changed.
c. Optionally, add a text field that holds a "diff -U" for ease of debugging.
2. Make a script on the deployed app that does:
a. Call (via RPC or RSS or whatever) the deployment server to find updated files.
b. Fetch the updates (via FTP or HTTP)
c. Check the updates for integrity (MD5/SHA)
d. Log the updates
e. Backup the to-be-replaced files
f. Optionally, install the updates or email an admin to visit a "finish my updates" page
3. Schedule said script to run nightly/weekly/whatever.
I rigged something like this up in under a day at one of the companies I used to work for. I think it was a grand total of 4 CF files:
- Nightly checker (deployment)
- Update checker (client)
- Script server (deployment)
- Finish-Your-Updates page (client)
It was unbelievably useful for keeping our multitude of clients in sync. Since you are only pushing files to production after you've tested them thoroughly (one would hope), you'll have very few breakages. It also makes you keep your code in one trunk instead of forking it off for every client.
Yes, there are security holes here and there, but nothing that can't be plugged with a bit of thought. Code signing, for example, was a bit heavier than we needed -- if someone was going to use DNS poisoning to get their files on our client's server, our client had bigger problems they needed to sort out. However, client authentication and IP verification on the deployment server were very easy to work in.
<project
name="projectName" default="build" basedir=".">
<target name="init" description="Set Props">
<!-- Define build vars -->
<property
name="project.name" value="projectName" />
<property name="project.build.root" value="c:\Build"
/>
<!-- Define destination targets -->
<property
name="dest" location="\\server.com\c$" />
<property name="dest.www"
location="${dest}\Inetpub\wwwroot\${project.name}" />
<property name="dest.inc"
location="${dest}\Inetpub\include\${project.name}_inc" />
<property name="dest.com" location="${dest}\Inetpub\com"
/>
<!-- Create the time stamp. -->
<tstamp>
<format property="buildTime"
pattern="yyyyMMdd_hhmmss_aa" />
</tstamp>
<!-- Define svn data. -->
<property name="svn.rooturl"
value="svn://server.xyz.com/" />
<property
name="svn.username" value="yourUserName" />
<property name="svn.password" value="yourPassword" />
<property name="svn.projecturl"
value="${svn.rooturl}${project.name}/trunk/" />
<property
name="svn.comurl" value="${svn.rooturl}com/" />
<property name="svn.tagurl"
value="${svn.rooturl}${project.name}/tags/${buildTime}/" />
<property name="svn.revision" value="HEAD" />
<!-- create build dirs -->
<mkdir dir="${dest.www}"
/>
<mkdir dir="${dest.inc}" />
<mkdir
dir="${dest.com}" />
</target>
<!--+++++++++++++++++++++++++++-->
<target
name="cleanBuildDir" description="Cleans the build
directory">
<delete dir="${project.build.root}" />
</target>
<!--+++++++++++++++++++++++++++-->
<target name="checkoutBuild" description="Pulls code from
Subversion into the build directory">
<exec
executable="svn">
<arg line="export
${svn.projecturl} ${project.build.root} -r ${svn.revision} --username
${svn.username} --password ${svn.password}" />
</exec>
<exec executable="svn">
<arg line="export
${svn.comurl} ${project.build.root}/com -r ${svn.revision} --username
${svn.username} --password ${svn.password}" />
</exec>
<exec executable="svn">
<arg line="copy
${svn.projecturl} ${svn.tagurl} -m 'Copy To Tag' -r ${svn.revision} --username
${svn.username} --password ${svn.password}" />
</exec>
</target>
<!--+++++++++++++++++++++++++++-->
<target
name="copyWWW" description="copies www files">
<copy todir="${dest.www}_bu" overwrite="true">
<fileset dir="${dest.www}" />
</copy>
<copy todir="${dest.www}" overwrite="true">
<fileset dir="${project.build.root}\wwwroot">
<exclude name="build.xml" />
<exclude
name=".project" />
<exclude name=".psd"
/>
<exclude name=".doc" />
<exclude name="test.cfm" />
<exclude
name="**/sql**" />
</fileset>
</copy>
</target>
<!--+++++++++++++++++++++++++++-->
<target
name="copyInc" description="copies inc files">
<copy todir="${dest.inc}_bu" overwrite="true">
<fileset dir="${dest.inc}" />
</copy>
<copy todir="${dest.inc}" overwrite="true">
<fileset dir="${project.build.root}\inc">
<exclude name=".project" />
</fileset>
</copy>
</target>
<!--+++++++++++++++++++++++++++-->
<target name="copyCom"
description="copies com files">
<copy
todir="${dest.com}_bu" overwrite="true">
<fileset dir="${dest.com}" />
</copy>
<copy todir="${dest.com}" overwrite="true">
<fileset dir="${project.build.root}\com">
<exclude name=".project" />
</fileset>
</copy>
</target>
<!--+++++++++++++++++++++++++++-->
<target name="build"
depends="init,cleanBuildDir,checkoutBuild,copyWWW,copyInc,copyCom"
description="Builds Project" />
</project>
I could be misinterpreting your comment, but I wouldn't agree that people shouldn't investigate other deployment strategies just because they're already using FTP and source control. There could be some other application or strategy that neither you nor I know about that would be well worth the time to explore. Now, making your deployment overly complex and administration-intensive-- that's overthinking. But you'll never know unless you investigate a little.
An automated deploy script would be just as beneficial - how many times have you FTP'd files and forgot something? Once that is scripted you would no longer have to worry about that.
Tim - thanks for the example script - that's going to get me motivated to RTFM for ANT.
1) can ANT use FTP or SSH for moving files around?
2) I primarily using CFEclipse with the Subclipse plugin to code and manage SVN - is there an ANT plugin?
http://ant.apache.org/manual/
There is no need for a plugin becuase ANT is built into Eclipse (though you can also download and run it separately if you wish).
Depending on the version of eclipse that you have installed (I always use the one thats about 100MB), ANT is all ready built in. I guess the easiest way to check is to create a build.xml file. If you see an ant icon on that file (literally a picture of an insect).
When you right click on this file, you should see a "Run AS -> ANT Build" option. Select that and watch the magic happen.
works great, but I am thinking of using Ant eventually for this...Ant is just uber slick.
...and we have a site wide license of BeyondCompare, as said above, this thing is slick and powerful.
DK
I put the build.xml file directly in the webroot. Notice this in my script : <exclude name="build.xml" /> . This command prevent the file from being copied to production.
We only "deploy" to one spot: The production web server. Deploying to some staging area and copying up seems redundant to me.
----
Ray, please keep posting discussion topics. You don't have to have the answers if you can moderate a discussion. I learn most from blogs that invite me to participate.
To do this though, you must fully use the trunk, tags, and branches methodology for Subversion. Since trunk is always our best working version, we deploy from the trunk. Others who do more 'scheduled' release might 'switch' using a branch or tag path.
I have a large web site with a complex directory structure under version control and the site requires frequent updates and deployments via FTP. Using a shell extension like TortoiseSVN, it isn't a pain to visually locate one or two files that have changed and upload them to the production server but this process becomes messy and error-prone if many files in many different directories have been changed.
How has someone else dealt with this successfully? Just do a bulk upload by selecting complete directories and hoping that your FTP program of choice can filter out all of the hidden ".svn" specific files? Is it possible to export unversioned from the repository only the files that have changed since a last revision while also preserving the directory structure if necessary?
I may be making this way more difficult than it needs to be but am interested to hear what others have done...
Thanks a ton,
Tom
I recently discovered Ant (I had heard of it before, but never really knew what it was or what it did) and hope to start using it in my deployment process.
In years past, these clients developed on a Production server, but when the server's lease expired, we changed it out for two servers - one development and one production.
Now we're working through the process of moving a users coded from development to production. At the moment, we're doing it the old fashioned way -- they call me, and I move it for them, but that's getting really old, really fast.
We don't want to grant the users access to the prod box because we don't want to tempt them to change code there directly. Often times, these sites update several times a day as new Excel or Word reports are published or vacation calendars are updated.
For some users, we've set aside space on a file server for such content and used IIS virtual directories to include the info simultaneously in both the dev and production without having to move anything around.
Both servers are Windows 2003, running CF7 and IIS 6. Our backend is SQL Server 2000 and we're not using any sort of version control. Just curious to know if anyone has any suggestions on how best to accomplish the goal of moving data from one environment to the other, automatically.
Thanks,
Bob Weston
bobwestonATsbcglobalDOTnet
I wanted to follow up on this post and say I've been using ANT now for several months to move my files from development into production and it's been working great.
I have lots of Ant info on my wiki if anyone is interested:
http://www.thecrumb.com/wiki/Ant
I am getting below erorr while deploying CF6 CAR file into CF 8
We are planning to migrate from Coldfusion Mx 6.1 to Coldfusion 8 While we deploying the CAR file created from MX 6.1 into Coldfusion , None of the SQL datasource is working . Log file containts the below erorr message Error occurred while updating datasources:tedy" "Error","jrpp-10","08/20/08","13:49:36",,"An error occurred while trying to encrypt or decrypt your input string: com.rsa.jsafe.crypto.dr: Could not perform unpadding: invalid pad byte.. " After adding this in JVM config below error occurs -Dcoldfusion.disablejsafe=true Complete JVM Args java.args=-server -Xmx512m -Dsun.io.useCanonCaches=false -XX:MaxPermSize=192m -XX:+UseParallelGC -Dcoldfusion.rootDir={application.home}/../ -Dcoldfusion.classPath={application.home}/../lib/updates,{application.home}/../lib,{application.home}/../gateway/lib/,{application.home}/../wwwroot/WEB-INF/flex/jars,{application.home}/../wwwroot/WEB-INF/cfform/jars -Dcoldfusion.libPath={application.home}/../lib -Dcoldfusion.disablejsafe=true An error occurred while trying to encrypt or decrypt your input string: Given final block not properly padded. "


Very user friendly, and has automated processes.
I make a copy of "everything that's changed since yesterday".
The more you read the manual, the more things you can do with it.