In honor of the 7th anniversary of the release of the classic strategy title Civilization V, we present this classic postmortem, which first appeared in the March 2011 issue of Game Developer magazine. The backstory of the game is fascinating: When the team began work on it, the technology was not yet in place. This forced them to run a development plan that put design and engineering on separate but simultaneous tracks. It wasn’t until the final year of Civilization V‘s almost four year production that the two came together. This in-depth look at what went right and what went wrong during development was written by lead producer Dennis Shirk.
Almost four years ago, John Shafer laid out his vision for the next iteration of Sid Meier’s Civilization to our prototype team (which consisted of Jon, seven artists, and the ubermoddable Civilization IV engine). Some of the more exciting features proposed at the time included sweeping changes like one unit per tile, hexes, complex full-screen leader environments, and a new scale involving more units onscreen than we’ve ever displayed before in a Civilization game.
On the engineering side, the excitement surrounding the creation of a new engine to accommodate these systems had an amazing effect. The entire engineering team had the opportunity to create something unique, built from the ground up for Civilization V. On the art side, the team was challenged to create a completely believable world, and to produce leader scenes composed of fully fleshed-out characters greeting players in their native languages. We were setting out to create a completely new Civ experience, which got the team excited to bring the design to life. It’s been a long road from prototype to final product, and as the vision was implemented, the challenges of delivering new concepts built on a new engine started to present themselves—but never in the places we expected them.
Two of our major goals for the project were to support ambitious new gameplay changes (one unit per tile, hexes, and so forth), and to elevate our target for the visuals. The first priority was obvious. We were going to need to create an entirely new graphics engine to take advantage of features we wanted to use from Direct X 11. Given our schedule, this plan meant that our new engine wouldn’t come online until 18 months before release—far too late for us to start testing these gameplay ideas.
Our solution was to enable a parallel development track for gameplay using the existing Civilization IV engine as the graphics component. We needed to keep a very clear interface between gameplay and the engine so that we could do a quick swap of engines without having to halt development on either side. In the end, we were able to run gameplay with both engines for a few months as the swap took place, which ensured as seamless a transition as possible. Once we had everything back together in the new engine, we already had a game that had been refined for almost two years in its Civilization IV incubator.
In January of 2009, our engineering team was still hard at work creating a completely new engine, including a custom renderer, for Civilization V. Broad testing across many different hardware platforms was something we had identified as a risk early on. As many of you may already know, PC development is tricky, to say the least. You can test on 500 different combinations of hardware, but once you release the game to millions of fans, your careful testing has a tendency to go out the window.
ATI, Intel, and nVidia were all instrumental in making sure this risk was minimized. Intel brought an engineer on-site to assist our graphics team with optimization for the new Core series of processors, and provided us with Sandy Bridge hardware. nVidia sent an engineer to work directly with us to implement its 3D solution, as well as help us optimize for the hardware.ATI not only helped us with optimization, but also provided us with cutting-edge AMD systems for testing. Both nVidia and ATI made sure that we had enough video cards for the entire team so we could test on both older and newer hardware in-house, and most importantly, both gave us access to their amazing compatibility labs. This was instrumental in minimizing hardware compatibility issues when we launched.
That’s not to say there weren’t problems. As with any new engine, there were issues that needed correcting once we released the game into the wild, and our venders were there with us, constantly updating their drivers, and continuing to make the end-user experience better with each iteration.
One of the new tactics we employed on this project was the co-location of our entire project team. From the start, the whole team sat on the same side of the building to increase ease of access to all teammates. Further, each smaller discipline—such as concept artists, modelers, animators, and the like—shared offices. Not only did we group certain disciplines together, we also clustered together people that frequently interacted with each other. For example, the lead designer, lead programmer, and lead artist shared an office. A graphics programmer was paired with certain artists to ensure that the group’s technical needs were met and that the artists followed protocol. This daily face-to-face interaction improved communication among team members. Access to quick answers from coworkers sitting close by allowed the team to problem solve and overcome obstacles in a timely manner, which helped us reach our goal.
Office culture is defined by the people who work in a certain setting. One department or office may have an overall culture, but if you look beneath that, groups who sit together frequently form their own microcosmic tribe. In a video game company, you often find a clear dividing line between artists and programmers. When all artists sit on one side of the building and all programmers on the other, tensions can build between the two disciplines.
By simply arranging the office by project, the walls between artists and programmers begin to break down and communication improves. Instead of setting disciplines against each other, aligning people by project tends to reduce tribal behavior, encouraging people to be loyal toward the project they are working on rather than to the discipline they are working within.
Furthermore, there are other benefits that go largely unnoticed, such as informal conversations that naturally happen in an office setting throughout the day. What may start as an offhand comment can lead to incremental adjustments. Information and knowledge is often shared unofficially among those individuals who sit together. Decisions are made in these day-to-day interactions and are often not passed on formally because they sprung from casual conversation. Therefore, by allowing team members to sit among one another, we avoided redundancy and unnecessary backtracking as decisions became formal throughout the course of a day, a week, and a project.
We had an experienced art team with a diverse skillset, which allowed us to solve problems quickly and effectively. At the outset, strong generalists were able to concept, model, and prototype their ideas quickly. This gave us a tremendous amount of flexibility to test concepts without a huge expenditure of man hours, and allowed the game designers to have a good idea of the visual direction we were taking early in the project.
Creating a unique title like Civilization requires a different perspective on iconography and problem solving. We had an advantage in that the majority of our artists had worked on previous iterations of the series and had good ideas to build upon. The interface lead incorporated many of the lessons learned from the console development of Civilization Revolution. The concept artist had spent much of his career learning the costuming and design from different eras in history, and took the opportunity to show off his vast knowledge.
We also benefited from having members of the team that had been lead artists at Firaxis in the past. These were artists who could be relied on to meet deadlines, be mature in conflict resolution, and be responsible with large aspects of the game. The project’s art director had a wise and seasoned group of advisors upon whom he relied upon to point out mistakes and to lend a hand to help fix them. They were also understanding of the conflict between visual direction and needs of the game design, keeping morale high when compromises needed to be made.
Complementing the learning from this seasoned group were the junior artists. Their excitement level about being in the games industry, and getting an opportunity to work on a franchise as important as Civilization, also helped motivate the team during production. The passion and creativity of artists desiring to make their mark gave our interface exciting illustrations of the icons as well as the memorable landscape of our game.
We also had a strong variety of people that came from other art disciplines. We had an artist with a film background, one that had studied industrial design, and another that came from traditional 2D animation. Combining this diversity to achieve a singular goal made this a fun team to work with, and they were also effective at fitting the history of civilization into a single game.
When Jon Shafer originally laid out his plans for the modding systems in Civilization V, everyone on the team was excited by the prospect. With past versions, modding was extremely popular with our hardcore fan base, but most of our casual players never even knew that many of these epic mods existed. Enter Shaun Seckman, our modding lead.
The systems he created and implemented will forever be used as an example of what we need to have in any game where we consider modding to be important. The system we designed allows any person playing the game to search for and download mods directly into the game. There’s no restarting, no technical knowledge needed, and it’s simple to use. Marry this with the tools that we created, like the standalone World Builder, and suddenly anyone can become a scenario designer. As a result, our download numbers for mods have already surpassed the million mark, something we could not have imagined when we released the game. Special kudos need to go out to GameSpy and their Special Projects team. They provide the back-end server infrastructure that makes it all possible.
Civilization IV: Beyond the Sword was as fleshed out a Civilization title as one could hope for. Expectations for a new version of the game would be extremely high, especially among our hardcore fan base. Since this was the fifth iteration of Civilization, our team came to the drawing board looking to do something profoundly new with the series. Our vision for Civilization V included many risky changes that would require a significant amount of new tech, and an even larger role for design and gameplay than in past versions.
The design radically changed three of the four types of victory from previous versions, and while this was exciting to us on paper, the challenges of designing and balancing it were numerous considering the schedule we had to keep.
One unit per tile was perhaps the biggest, most noticeable change. Whereas a player in previous versions would work with large stacks of units, one unit per tile was more about expanding the tactical game to make it more interesting and engaging. While I think we succeeded in this concept, the time commitment to this system needed by our design team was fairly costly, and it had a very real impact on the other core components of the game. An entirely new AI system also had to be created, and while great strides were made, we underestimated the time needed to make such a large system work in a consistent, competitive manner.
The reality is that the more we focused on brand-new systems to create a brand-new experience, the more we had to trim systems that players had come to expect from previous versions. We ultimately had to focus on making sure our core systems and new concepts were working well, sacrificing some of the less critical features. Some of our hardcore fans have been disappointed by the lack of certain features, but this prioritization has given us a solid foundation to build on, and we’re restoring or improving most of that functionality and more, as we continue to support the game moving forward.
During Civilization IV‘s development, we utilized an external design group called “Frankenstein,” which was primarily made up of some of the most hardcore fans of the series who know the game inside and out. Once they received NDAs, we passed them regular builds, and they provided extensive gameplay testing and feedback to the design team. For Civilization IV, we strongly believe that the working relationship between the Frankenstein group and our team was one of the key reasons for its success.
For this reason, we set up a new team with community veterans from the previous team, along with some new additions, and Jon started working with them early in the original Civilization V prototype process. At this point, we were still delivering builds that used the Civilization IV engine married to Jon’s new game core. Because the engine had already been released DRM free, there were no issues pushing out regular builds to our external testers. The issues cropped up when the new engine was finally ready to make its debut.
When we were finally ready to move the game core to the new engine, our DRM solution (via Steam) was not yet approved, nor was it integrated into the build system. Because this was a brand-new technology, there was significant work needed to get it to a place where we felt comfortable allowing the builds to start propagating out to our testers. The unfortunate thing is that the implementation took close to two months. Two months with no new builds going out to our external gameplay and design team. Two months with zero feedback. For a game that needs a tremendous amount of balance to perform well over the course of a 12-hour session, this was a painful process to go through.
Ultimately, we did get the external team back on track, but the lost time could not be recovered. Thankfully, Frankenstein is filled with possibly the most dedicated people on the planet. Civilization, for them and for us, is something that can never be left “as is.” They’ve never stopped working, and continue to provide invaluable feedback that makes Civilization V a better game, even post release. As we march through the DLC process, we’ve put in place an aggressive patch schedule that allows us to incrementally improve almost every core mechanic of the game.
To day ’s game development environment is heavily focused on the multiplayer component. Facebook is huge and MMOs are going strong. As a result, finding qualified networking programmers has become akin to spotting a unicorn in your backyard. One of the biggest challenges we had to overcome was not having a staffed-up multiplayer team until well into production. We were fortunate to have a solid example with Civilization IV, but the amount of gameplay changes coupled with a completely new engine meant that much of it had to be coded from the ground up.
This is an area where 2K QA and our internal QA team and engineers adapted very well. With the compressed timeline, we had to put together an aggressive testing schedule to get multiplayer functioning well and ready to ship. Once we had core functionality set up, the multiplayer play sessions became extremely important. We organized a strike team composed of our networking engineers and two gameplay engineers to float around the office during the sessions. This way, as individuals ran into out-of-sync issues, we were able to identify the exact nature of each problem, correct the problem, and deploy new builds quickly.
While this successfully got us to a point where we were able to ship the game, the multiplayer experience was lacking many features that were present in previous versions of Civilization. We absolutely do not consider Civilization V’s multiplayer to be a “closed book,” and as with other aspects of the game, we are continuing to improve the experience to meet our standards, as well as those of our fans. And for the record, I think our engineering team still holds the edge for “games won” over 2K QA. I’m not, of course, including the last session where I was knocked out in 30 turns by a horsemen zerg.
Because of the amount of attention the new combat system demanded, a significant amount of time was spent fine-tuning and iterating this concept throughout the early eras of the game. As changes were made, new games were started and concepts were tested. The problem this introduced is that Civilization by its very nature is a long and involved game. The time requirements to test a game like this are significant. You cannot just test a single system, you have to constantly test to make sure the system works throughout the length of an entire game. What may work wonderfully for the Ancient era may not work as well for an Industrial or Modern era. While we do have the ability to start a game in the Modern era, for purposes of balance and gameplay, there is no substitute for playing through a full game.
Ultimately, there ended up being a large disparity between the amount of playtime invested in the first half of the game versus the time spent testing the second half of the game. When you’re early in development, each new build had the potential to break earlier saves, so testers frequently had to start over from scratch, not always able to complete a game before the next build would rear its head. Because of this, there are some imbalances that were not revealed until the game made it into the hands of our fans. It really reinforced the notion that above all, we have to find a way to make sure that save file compatibility from build to build is always maintained. This can be really difficult when you’re in the middle of design, but we learned a hard lesson about what can result from this.
You cannot underestimate the effect a layoff has on team morale, especially when it lands in the final weeks of Beta. The realities of last year’s recession unfortunately touched our team when we were at our busiest, trying to finalize all the features and submit our Gold candidate. We lost some critical team members. It’s a situation that you can’t possibly prepare for.
Dev teams become a tight-knit group three years into the making of a game, so there is lost productivity as people adjust to friends having lost their jobs. Bouncing back from this was challenging, but to the team’s credit, we were able to regroup and refocus on the work that needed to be done. In the end, through some very hard work and extra help from our other development team, we were able to hit our street date and deliver a quality game. So, I suppose this can be considered both a What Went Wrong as well as a What Went Right.
We’re incredibly proud of what we accomplished in Civilization V. We overcame significant challenges in production to ship a critically acclaimed game on time, and one that is both completely moddable (with more content arriving every day, and DLL source code on the way), and that has introduced Civilization to a new generation of players. We’re aware of the high expectations of our fan community and are glad that overall, they feel we’ve delivered another great CIV game. We’re committed to supporting the game as we move forward and are thankful for the support and feedback from the community in that effort.