THE REAL DEAL with the HL2 source

G

gloryofbach

Guest
THE REAL DEAL with the half Life 2 source

I've seen a lot of confusion about what the Half-life 2 source is and isn't, whether it can be compiled into a working copy of the game or not, and whether this is bad or not for Valve.

I am a small time games developer myself and for the benefit of those in the community not into software programming or knowledgeable in games development, I thought I'd try and give a brief explanation. Apologies to the experts as some simplifications have been made for general consumption.

1) A game like Half-life 2 consists of two main elements, resources and programming code.

The resources consist mostly of all the non-logic/instruction based elements of a game, that is the art, textures, sound, story script, maps etc. Things like the model and textures for the headcrab is typical of this. This is the largest part of a game in size, for a game like Half-life 2 typically over a gigabyte of the game will be resources. These resources cannot do anything by themselves, and if you have the right software, can be viewed/played as when making a mod. These are like the bricks and wood for the "house" Valve was building.

The program code consists of the logic and instructions as to how the game will function. That is to say, the code directs the computer in how to utilize the resources mentioned above, in the correct manner to allow you to play a game. Examples of what would be included in the program code are the Source engine, the Havoc engine for physics, artificial intelligence routines, game launcher code etc. Using the analogy above with the headcrab, the corresponding program code would be the animation rules for the headcrab, its artificial intelligence behavior, its stats etc.
This is like the architectural plan of the "house" Valve was building.

2) To play the game (as you will when it is released) you need the program code and the resources. What has been leaked is the program code, which amounted to around 160 megabytes of text (though this did include some non half life 2 related code), which is a SERIOUS amount of code. The resources for half life 2 are not included, and they would have amounted to at least a gigabyte.

When programmers write code, they write it in so called high level language like C++. A high level language is one that is easy to understand to humans, so that editing it is easy during development. This is known as the source code, and is what Valve will be working on right until half life 2 goes Gold. This is what has been leaked to the public. The source code has many lines of C++ code, which you will have trouble understanding if you do not know C++ (though it is very easy to learn!), and some comments that explain what the code is doing.

A computer is not designed to directly execute C++ code, which has been designed for a human to read. Instead it executes machine code, a far lower level language which is very hard for a human to understand. Coding in machine code would be a nightmare for a programmer and so they prefer to use C++ and other high level languages. Now for the computer to understand Valve's C++ source code, it must be converted to machine code. This is what a COMPILER does. Valve will run a compiler on the source code every so often, creating a bunch of executable files (machine code based) that can then be used to run the game and test the changes made to the source code. This is what you get when you buy a game: Resources and the compiled source code to run them.

As such, the fact that the source code is in the public domain does not mean that anyone can compile a working game out of it, as all the resources are missing as well as a few other code elements. So this source code leak isn't a big deal for Valve right? Why all the worry?

3) A source code leak is a HUGE deal. Source code is only released for obsolete games (like Quake, Quake 2 etc) once the programming secrets they contained are deemed out of date and no longer valuable to rival developers.

Half-life 2 is NOT an obsolete game. Is a cutting edge game incorporating many features that have never been seen before in a game, and more importantly is HAS NOT BEEN RELEASED. What does this mean for Valve?

It means all the work behind the new features, the Source engine, the revolutionary AI etc is now exposed for anyone in the public and rivals to see, learn from, and God forbid, copy. Using the house analogy above, it means the new revolutionary solar powered house you were building has had its building plans stolen before you've built it. Now everyone knows how you solved all those problems which prevented people from building revolutionary solar powered houses.

As an example, the Strider as seen in the released videos, is an amazingly novel creature to see in a computer game. It is seriously big, yet maneuvers around and interacts with the landscape in a convincing and intelligent manner. Getting a Strider to function like this will have taken Valve months of painstaking AI and animation work. Rival developers would have had to guess how on earth Valve pulled it off. Now they can simply look in the equivalent of the Strider AI source code file, and it will all be there, conveniently commented and explained by the Valve programmers for their own internal private understanding.

The source code for half life 2 is the result of 5 years of hard work by some of the best programmers in the games industry. No one would have seen it (bar engines licensees) for 5 or 6 years at least, upon which it might have been released to the public (like Id software with Quake).

The source code is not to be confused with the SDK as some people have. The SDK for half life 2 which will be released, is its Software Development Kit. This allows modders and fans to create games and mods using Half-life 2 technology, but does not tell them how the technology works. Unfortunately this is exactly what the leaked source code tells people.

So in short :), this means for Valve:

All its clever routines and algorithms for making a revolutionary game like half life 2 tick, are available to be understood by anyone who knows C++, including game development rivals, hackers making cheats and exploits and the general public.

More worryingly the source code seems to contain a lot of source code from partner software developers that Valve had licensed to use in Half- life 2, such as the Havoc physics engine, and that Valve is contract bound to keep protected and confidential. These leaks enable people to potentially rip off these software partners too, and hence opens Valve to be sued.

Finally the internal workings of Half-life 2 have been so seriously compromised that I will be amazed if we see it released before 1st Quarter 2004. The potential for hacking exploits is enormous; it is like printing the blueprint to your high security safe in the local newspaper before having it installed in your home. SERIOUS work is needed now to rework the source code to protect it from future attacks on release. Let us not even mention that it seems Steam has been seriously compromised as well in the leaked source, as that had enough problems of its own to be getting on with without this.

A leak of the source code of a revolutionary highly anticipated game like Half-life 2, close to release is, as far as I know, unprecedented. My deepest sympathy as a games developer goes to the Valve crew and I wish them every success in catching the perpetrators of this crime.

Thanks for reading!
 
Nice post...

btw, i have never seen so many words in one place. My mind boggles.
 
finally, someone thinking about something other than possible hacks etc. The privacy of the data is a MUCH larger issues, for valve and the community.
 
Thanks for the comments guys, people need to know what a big deal this is and support Valve and Gabe in their time of need!
 
Nice read so the code was leaked on the 19 of september is this the reason why the game was and is delayed then?
 
No need to post that, apparently 97% of the board are all C++ gurus judging from their spout on all the other threads about the sourcecode.
 
Nice article, But most of this stuff I read in a book called SOFTWARE SYSTEM DEVELOPMENT.

but anyway nice copy paste because it fit the subject.
 
Haha thanks Gorgon!

BTW this wasn't a copy and paste. I've yet to read a nice concise explanation of coding so I wrote my own here!!! Took me half an hour too!!!! :)
 
great post m8.l Interesting read.

I understood all the concepts before. I aint a c++ programmer though either :D

Thx for the post very interesting
 
Re: THE REAL DEAL with the half Life 2 source

Hello,

I'm new to this forum and got sent here by a friend in the Raven Shield community, thanks Corn-Flex. I really can't wait to get my hands on Half Life 2, since all the eye-candy sent out so far is just awsome. I'm also a hobby game programmer, and I know how much work is invested into these games. But I don't agree with you totally gloryofbach. Say the source code is 1.000.000 lines of code, which is not that unrealistic. That is ALOT of code. Say you read a book with 100 lines on each page, then the book would contain 10.000 pages. That is a huge book that would take a very long time to read. Now take into account that you also have to learn game logics, concepts, the overall structure etc. etc. etc. That will probably take almost as long as writing the engine yourself. Now think about the fact that you have to learn the code, understand it and then turn it into something that's not completely Half-Life 2 like. By the time they release a game from the stolen source code, the Half Life 2 engine will probably be obsolete.

The worst thing imho is that hackers get their hands on it and can create cheats etc.
 
@Deficite - No need to read the whole book if you are only interested in some chapters. AI Coders would be interested in the AI codes for the HL2 enemys, 3D Engine Coders would only check the core 3D Engine and so on. No need to read a whole book if you are only interested in the foreword.
 
One other item of note:

No the game resources were not stolen and hence the game is unplayabel, however, once the game IS released, those resources can easily be copied from the gam CD and a fully functioning copy of the program would be available as warez with any authentication required being easily disabled by minor code changes.

Valve will have to re-write at least enought to prevent this from happening. I woun't even go into the ramifications of the Steam source being leaked along with it. Needless to say, I'm sure this will need MAJOR changes to stay secure with the plans Valve has for it.

My $0.02 anyway.
 
frischo is absolutely right dude, you're not gonna read the whole 'book', you'll skip to what interests you. And cheating is the last thing im worried about. I'm worried for the sake of the future of valve and their engine technology. Lets not forget, leaked with this sources was a lot of code which was lisenced from other people, eg Havoc, and valve is open for a potential lawsuit with the leak.
 
gloryofbach,

Are you saying, that when they would release the game at this stage; the resources would become available... and therefore giving everyone complete control of valve's hl2 technology?

I guess when people have access to the resources (featured on the hl2 release-cd's) they will be able to fully understand and use the source code.

In that case maybe there's even a spark of light in this shit:
The incredible technology is open-source now, so it could be that the games development world will be making a leap forwards in general at the expence of Valve.
 
I would be shocked if Valve actually kept game source code on ANY machine that had any connection at all to the internet. I would think that any game development machines they had would be networked together ONLY with no network connections at all to any other machine that could reach the outside world.

If Valve indeed DID have any outside connectivity from their development boxes to the outside world, then shame on them for being so security unconscious.
 
That still doesnt justify it.
Just because i left my fronto door open doesnt mean you can come in.
 
So, Basically VALVE have to Rewrite all the CODE in HL2 ??

and ..may be
Called it HL2X which could be even better than HL2 originally //
Most of all ..Valve has lots of work to do with this ..
Just Like DOOM III demo leaked .. PUSH DOOM III to Next Year..

What will happen to HL2 Then? Release date Next X'mas --2004 ??
OH No..that's not good.;( :borg: :x

Hope VALVE can figure it out and have it ready this X-mas..

Don't worry ..there will be a better tomorrow..
:cheers:
 
Theunage and everyone else,

It is safe to say that the leaked source code is THE reason Valve has delayed Half Life 2 indefinitely. All this talk of Steam and internationalisation issues is a smoke screen.

Valve knew about the source code files being copied from a Valve internal computer soon after it happened, and must have been praying/hoping that the attempted steal was poorly exectuted, or that the hacker wanted it for private consumption. In the mean time, a source code leak is so potentially serious that they MUST delay Half life 2 just in case. As they did.

In the mean time running up to yesterday, Valve PRAY the leaked source does not go public domain.

The leaked source goes public domain.

The shit hits the fan for Valve, and now everything Half-life 2 related is in doubt. This holiday release date rubbish is a place holder while Valve do serious damage control. Again don't expect a pre 2004 release.

Thanks.
 
Originally posted by Tropico
That still doesnt justify it.
Just because i left my fronto door open doesnt mean you can come in.

I am by no means condoning the source theft. Leaving the front door of your house open is only inviting theft, not condoning it. It's wrong but certain safeguards need to be maintained especially with other peoples (Havock) source you've been entrusted with.
 
Thank you for the post, I was honestly wondering what the big deal was. But now I see, and I thank you for that. Now on a side note, I wonder how this will affect the development cycle. Some people are saying release it now, while others are saying wait a year. I think that this is all just wild speculation, such as all of the HL2 was delayed because aliens from one of pluto's moons meant to be performing cow mutilations in the midwest, but got the wrong coordinates and ended up at VALVe where they killed a few employees, then feeling remorseful they gave Gabe a peice of thecnology that would save the world if we could only understand it's uses, and this technology was STEAM, but VALVe had no clue what it was for, so they stared working on modifying it to distribute all of their games, which is why HL2 isn't out yet because alien's don't code in C++, they code in HsdlfjJDf lkjfsijf idsfjsdiljf++, so in reality don't expect HL2 until 2022. I know that this was one of the more popular theory's running around here for a while.
 
Originally posted by incorrigible
I would be shocked if Valve actually kept game source code on ANY machine that had any connection at all to the internet. I would think that any game development machines they had would be networked together ONLY with no network connections at all to any other machine that could reach the outside world.

If Valve indeed DID have any outside connectivity from their development boxes to the outside world, then shame on them for being so security unconscious.

I work and make money from software development. We don't make hundres of millions of dollars like Valve from their games, but this totally surprises me. How can a company with such an unmeasurably valuable source code have this on a compute that is physically connected somewhere to Internet, I am speechless almost. Nop matter how secure systems you have, I don't see CIA or FBI or NSA etc have their interla systems physically wired to the world. They are physicall separate and physical measures like key cards, finger print scanners etc are required to login to a computer on that secure separate network.

I'm not saying Valve should have had that kind of security, but the source code should have been unavailable physically from Internet. I just can't beleive it, especially not Valve. I mean compiled betas leak everywhere, ever from iD, and Valve has been able to keep *everything* secret so far, not ever any mention of multiplayer, weapons etc, thing that a Valve developer could leak over a few beers in a bar even. NOTHING has come out of Valve in all these years and now the complete source code is available via an Internet hack... I don't know what to say!

I think one thing is for sure, it will go down in history, it will make front page in newspapers in a few days etc, it will be rememberd for years to come, it might even shift the whole gaming industry.
 
Some posters have said there are millions of lines of code, and that to understand them without the resources would be impossible.

Unfortunately, all responsible programmers extensively doucment and comment their source code, to remind them later what everything does. This makes understanding source code much easier. I have had the chance to look at some of the leaked source of a Friend's computer, and I am sorry to say to a programmer it is very understandable.

It took me 10 minutes to understand the Strider (as I used in the above example) AI, how it looks for targets, the fact it strafes rather then pinpoint fires, the way it maneuvers under bridges. Its all in there.

As well as a list of vehicles that is very surprising.

Again I feel sorry for Valve and will not release any spoilers, but it is all there for people to see now.
 
This is an absolute nightmare. The game industry's September 11th.

;(
 
Just imagine what would happen if the Windows souce code was leaked :O
 
life sucks. I jsut wanted to play HL2 before I start college... my last great feat for summer.
For the first time I find myself pissed off... and hating HACkERS!

**** them all!
 
I'd have to say that I agree with incorrigible.
While leaving your own code on a machine with access to the net that runs on software with known security holes arguably is a pretty stupid thing to do, it still is your own code and you are solely responsible for what you decide to do with it.
Doing this with licensed code, however, is another matter entirely. In doing this, you are breaching your contract with the company you licensed the code from, endangering both them and your own company via potential lawsuits. This is irresponsible in the extreme.
I'm not saying that it isn't understandable... it's a human thing to do, and seeing as how Valve considers itself part of the community, they wouldn't have expected an attack like that (which was quite professionally done, by the way).
Anyway, finding someone to blame isn't what really matters now... Like gloryofbach said (thanks for the posts!), damage control is the one thing Valve has to concern themselves with now. Let's hope they have more luck with that than they had these last weeks.

Edit: Right, Pjotr already said it while I was still typing. Thanks Pjotr, full ack.
 
Originally posted by gloryofbach
[...]

You're absolutely right.
The source is very organized and well documented. U can just look up a .dll of an npc and see its entire behaviour scripts.

This is what makes me 100% sure this is the real source code.

Last night _MAXX_ some other guys and me have made a .txt file, under Munro's supervision containing all the spoilers relating to characters, monsters, weapon, locations, TF2, etc etc. It's Munro's decision what to do with the info. We WONT release any spoilers in the fora. And I hope nobody else will...
 
Originally posted by Dr-X
N0N1337H41 - lmao :D nice to keep thing light hearted

Thanks, everyone's acting like the sky is falling and I STILL don't see how it's going to have a big affect on me. I don't see why VALVe would have to wait years to release it, I mean, the damage has been done. As soon as some ass gets to looking through the code they'll post the secrets that we weren't supposed to know until we played the game. I'm sure it has to be a blow to the devs who spent years pouring their hearts into the project, but it's not the end of the world, people. I can forsee another month or two of flame wars having to do with whether this will delay the game or not. I suggest that everyone go pick up a copy of Star Wars Galaxies and flame eachother there while killing your favorite monters from the SW universe.
 
gloryofbach, very nicely written =)

For the people who say it's open source and everything - Valve can sue you (and win..) just for you looking at it. Let alone _using_ it.
 
The question floating around my head ever since the leak was annonced that I kept asking myself was, "what kind of security are they using?" I mean, you would think they would better protect something as important as that. If you ask me, I think they cheesed on their network security (either from ignorance or stupidity take your pick) and got burned for it. Its a shame, but I think valve might want to fire whoever was in charge of their security and spend some money to ensure it doesnt ever happen again.
 
How I think it happened

From what Gabe said, and my knowledge of source code control measures pre release, I speculate that the main reason behind this leak is Gabe's caring attitude to the community, and how it was viciously exploited by a heartless person.

Valve are NOT amateurs. Amongst games devolopers they are regarded as possibly the premier PC development crew at the moment. Everything they touch turns to Gold. They had serious source code protection measures in place. I expect every Valve developer has the source code on a seperate LAN based computer to the computer they use for internet comms, and all source code was identity marked in some way. This is standard control procedure for sensitive source code.

THE WEAK LINK WAS GABE. As the owner of Valve, he may not have been as careful as the rest of him employees (who would be fired if they compromised HL2) and allied to the fact he was constantly communicating with the fans over the internet, made him an easy target.

This wasn't a script kiddie job. This was a pre meditated attack by a pro with custom written hacking software to prevent early detection. Heres my timeline guess:

1) Gabe supports the community by answering emails regularly, clicking on humurous HL2 links sent to him by fans, maybe even opening up demo levels and other HL files sent to him by fans. He has commercial antivirus and security software installed, rightly expecting that an attack on Valve would not be executed by a "serious" hacker, to who retail security products are a joke.

2) At some point, Gabe inspects the HL2 source on his computer, he is a programmer nad may have comtributed small bits of code or optimisations, but as he is so busy with upcoming release, absentmindedly has it on his non secure internet connected PC.

3) Someone decides they want to screw Valve, and they formulate a plan and execute it coldly and efficiently. Gabe's computer is compromised by a custom written Trojan that has not been seen before, maybe via outlook security holes, or by Gabes habit of indulging community emails. Gabes computer starts acting funny, and at this point gabe should have worked it out, but he is very busy with other things. The hacker now has access to Gabes compromised PC via a trojan and can carefully over a week, determine Gabes schedule etc via his email access, to determine when best to download the source files he has found and whatever else he wants. Critically he has a long time (1 week it seems) before he is detected.

4) At some point the source files are downloaded via the trojan.

5) Valve internal control software logs an external unauthorised download, and Valve now know its rubber pants time.


I will not be surprised if Gabe decides to stop responding to community questions and Valve becomes even more secretive than it already is.

By the fact that someone wrote their own Trojan to get into Valve, and that it was executed so well, I expect this is by the kind of hacked that tends to target online banking websites, NASA, FBI and such like. The SERIOUS hacker, who is an excellent software developer in his own right. The problem is Valve is a soft target compared to the others mentioned and as such was easy pickings for a pro.
 
a thought gabe travels
gabe has a laptop
the laptop allows him to work while he travels
gets back to the office plugs into the lan and lwell the rest is history
 
Yes the laptop is also a major security leak waiting to happen especially when you are travelling to trade shows and ATi presentations etc, and you're tired.

Also he mentions he formatted his computer, but regardless, the hacker will have been able to pull of enough security information to allow him to compromise the reformated PC, not to mejtion he already had email access.
 
Back
Top