Compile Time = insane

[Dario D.]

My map has had this problem of long compile times for a long time now, but I'm wondering if something is wrong. I just downloaded the single-player campaigns "Penetration" and "Coastline to Atmosphere", and one of them (I don't remember which) said something like "compile time: 18 hours for all maps"... and what puzzles me is that the maps are very large, often with a lot visible at one time.

...On the other hand, my lone map, which is only medium-sized, takes about 3 days to do a full compile (which is compiling right now... and has been for the last 2 days).

To describe the layout of my map, it's similar to DM_Overwatch (circular, with a large building the center), but with 2 or 3x more polygons and details. There are about 5 main areas, and half of them have a good view of most other areas (making occlusion difficult, unlike Overwatch, which zones things up nicely).

I understand that this map is moderately more detailed and not as efficiently zoned as Overwatch, but I don't understand why Full Compile takes 3 days, when Quick Compile only takes 20 minutes.

Is something going on?

 

[Unarmed]

Something is indeed going on.

Did you ever map for CS 1.6? A compile of a map there took about 5 hours, if the map was done right. A compilation of a source map should never reach even that, if the map is done well, with a few exceptions.

The reason why your map is compiling so damn slow (And I do believe that this is something of a record-breaking compile time) is that you have not optimized your map. Luckily there is a very good tutorial to teach you how to optimize.

http://www.student.kun.nl/rvanhoorn/Optimization.htm

You need to read it all but you really need to study chapters:
1. general notices (saves on vvis-time and map speed)
2. nodraw texture (saves a bit on vrad-time and map speed)
3. func_detail (saves only a lot on vvis-time, can only decrease map speed if used incorrectly)
4. hints (and introduction to visleafs) (saves occasionally on vvis-time, a lot on map speed).

Happy optimizing.

//Unarmed

 

[UltraProAnti]

sounds like you've got some crazy angled surfaces not flagged as detail. try to make sure all your structural (i.e. non-detail) brushwwork is aligned with the grid, and there aren't any stray brushes floating around you forgot to flag as detail. If a non-aligned chunk of brushwork HAS to be structural (i.e. like in a cylindrical lighthouse), enclose the structure in a grid-aligned 6-sided block of hint. this stops the BSP cuts made by the angled surfaces from spreading out into the rest of the map (they stop at the hint surface)

 

[Dario D.]

Hmm, thanks for suggestions and the link. That tutorial has some interesting optimiztion methods that I never thought about (like rotating hint brushes 45 degrees at corners). I guess I can try some of them, though most of them are already in use. It might just be that my map is very open, and has a large scenery area (all func_details), but I would normally expect that to only increase compile time by several hours.

 

[Unarmed]

Something is wrong... Try running glview or whaterver its called to check how your vis is done. These compile times should not happen.

 

[Dario D.]

Yay! It finally finished.

Okay, this is what Portal Flow said when it was done:

PortalFlow: 0...1...2...3...4...5...6...7...8...9...10 (385813)

What number do you guys get at the end there? 385813 seems pretty huge compared to Portal Vis, which is 33.

-- Incase it contains valuable info, here's the whole compile log. I also don't know what to make of the list at the end, with all the percentages:

Valve Software - vbsp.exe (Aug  4 2006)
1 threads
materialPath: d:\half life 2\steamapps\dariodee\half-life 2 deathmatch\hl2mp\mat
erials
Loading D:\Half Life 2\SteamApps\dariodee\sourcesdk_content\hl2mp\mapsrc\dm_plaz
a17.vmf
fixing up env_cubemap materials on brush sides...
ProcessBlock_Thread: 0...1...2...3...4...5...6...7...8...9...10 (1)
ProcessBlock_Thread: 0...1...2...3...4...5...6...7...8...9...10 (1)
Processing areas...done (0)
Building Faces...done (0)
Chop Details...done (0)
Find Visible Detail Sides...
Merged 1456 detail faces...done (1)
Merging details...done (0)
FixTjuncs...
PruneNodes...
WriteBSP...
done (1)
writing D:\Half Life 2\SteamApps\dariodee\sourcesdk_content\hl2mp\mapsrc\dm_plaz
a17.prt...done (1)
Creating default cubemaps for env_cubemap using skybox materials:
skybox/sky_day02_02*.vmt
Run buildcubemaps in the engine to get the correct cube maps.

No such variable "$hdrbasetexture" for material "skybox/sky_day02_02rt"
Can't load skybox file skybox/sky_day02_02 to build the default cubemap!
Finding displacement neighbors...
Finding lightmap sample positions...
Displacement Alpha : 0...1...2...3...4...5...6...7...8...9...10
Building Physics collision data...
done (1) (702511 bytes)
Placing detail props : 0...1...2...3...4...5...6...7...8...9...10
Compacting texture/material tables...
Reduced 6082 texinfos to 3278
Reduced 603 texdatas to 510 (30489 bytes to 25892)
Writing D:\Half Life 2\SteamApps\dariodee\sourcesdk_content\hl2mp\mapsrc\dm_plaz
a17.bsp
27 seconds elapsed
Valve Software - vvis.exe (May 22 2006)
1 threads
reading d:\half life 2\steamapps\dariodee\sourcesdk_content\hl2mp\mapsrc\dm_plaz
a17.bsp
reading d:\half life 2\steamapps\dariodee\sourcesdk_content\hl2mp\mapsrc\dm_plaz
a17.prt
2478 portalclusters
7003 numportals
BasePortalVis:       0...1...2...3...4...5...6...7...8...9...10 (33)
PortalFlow:          0...1...2...3...4...5...6...7...8...9...10 (385813)
Optimized: 15958 visible clusters (0.00%)
Total clusters visible: 1972588
Average clusters visible: 796
Building PAS...
Average clusters audible: 2220
visdatasize:1318676  compressed from 1546272
writing d:\half life 2\steamapps\dariodee\sourcesdk_content\hl2mp\mapsrc\dm_plaz
a17.bsp
107 hours, 10 minutes, 52 seconds elapsed
Valve Software - vrad.exe SSE (May 22 2006)
----- Radiosity Simulator ----
1 threads
[Reading texlights from 'lights.rad']
[45 texlights parsed from 'lights.rad']

Loading d:\half life 2\steamapps\dariodee\sourcesdk_content\hl2mp\mapsrc\dm_plaz
a17.bsp
8572 faces
4072367 square feet [586420864.00 square inches]
12 displacements
130257 square feet [18757010.00 square inches]
8572 patches before subdivision
66867 patches after subdivision
78 direct lights
BuildFacelights:     0...1...2...3...4...5...6...7...8...9...10 (524)
BuildVisLeafs:       0...1...2...3...4...5...6...7...8...9...10 (290)
transfers 7252759, max 1092
transfer lists:  55.3 megs
GatherLight:         0...1...2...3...4...5...6...7...8...9...10 (1)
        Bounce #1 added RGB(393263, 303599, 201029)
GatherLight:         0...1...2...3...4...5...6...7...8...9...10 (0)
        Bounce #2 added RGB(63525, 45734, 28166)
GatherLight:         0...1...2...3...4...5...6...7...8...9...10 (1)
        Bounce #3 added RGB(11782, 7923, 4640)
GatherLight:         0...1...2...3...4...5...6...7...8...9...10 (1)
        Bounce #4 added RGB(2576, 1633, 864)
GatherLight:         0...1...2...3...4...5...6...7...8...9...10 (0)
        Bounce #5 added RGB(600, 361, 171)
GatherLight:         0...1...2...3...4...5...6...7...8...9...10 (1)
        Bounce #6 added RGB(149, 85, 35)
GatherLight:         0...1...2...3...4...5...6...7...8...9...10 (1)
        Bounce #7 added RGB(38, 20, 7)
GatherLight:         0...1...2...3...4...5...6...7...8...9...10 (0)
        Bounce #8 added RGB(10, 5, 2)
GatherLight:         0...1...2...3...4...5...6...7...8...9...10 (1)
        Bounce #9 added RGB(3, 1, 0)
GatherLight:         0...1...2...3...4...5...6...7...8...9...10 (1)
        Bounce #10 added RGB(1, 0, 0)
Build Patch/Sample Hash Table(s).....Done<0.0825 sec>
FinalLightFace:      0...1...2...3...4...5...6...7...8...9...10 (2)
FinalLightFace Done
Computing detail prop lighting : 0...1...2...3...4...5...6...7...8...9...10
0 of 3 (0% of) surface lights went in leaf ambient cubes.
ComputePerLeafAmbientLighting: 0...1...2...3...4...5...6...7...8...9...10
Ready to Finish

Object names       Objects/Maxobjs  Memory / Maxmem  Fullness
------------       ---------------  ---------------  --------
models                  19/1024          912/49152    ( 1.9%)
brushes               1906/8192        22872/98304    (23.3%)
brushsides           12759/65536      102072/524288   (19.5%)
planes                7056/65536      141120/1310720  (10.8%)
vertexes             21891/65536      262692/786432   (33.4%)
nodes                 4682/65536      149824/2097152  ( 7.1%)
texinfos              3278/12288      236016/884736   (26.7%)
texdata                510/2048        16320/65536    (24.9%)
dispinfos               12/0            2112/0        ( 0.0%)
disp_verts            1124/0           22480/0        ( 0.0%)
disp_tris             1824/0            3648/0        ( 0.0%)
disp_lmsamples       67675/0           67675/0        ( 0.0%)
faces                 8572/65536      480032/3670016  (13.1%)
hdr faces                0/65536           0/3670016  ( 0.0%)
origfaces             5759/65536      322504/3670016  ( 8.8%)
leaves                4702/65536      150464/2097152  ( 7.2%)
leaffaces            12663/65536       25326/131072   (19.3%)
leafbrushes          10326/65536       20652/131072   (15.8%)
areas                    2/256            16/2048     ( 0.8%)
surfedges            65666/512000     262664/2048000  (12.8%)
edges                43527/256000     174108/1024000  (17.0%)
LDR worldlights         78/8192         6864/720896   ( 1.0%)
HDR worldlights          0/8192            0/720896   ( 0.0%)
waterstrips           1242/32768       12420/327680   ( 3.8%)
waterverts               0/65536           0/786432   ( 0.0%)
waterindices         23184/65536       46368/131072   (35.4%)
cubemapsamples          89/1024         1424/16384    ( 8.7%)
overlays                27/512          9504/180224   ( 5.3%)
LDR lightdata         [variable]     3399948/0        ( 0.0%)
HDR lightdata         [variable]           0/0        ( 0.0%)
visdata               [variable]     1318676/16777216 ( 7.9%)
entdata               [variable]      119649/393216   (30.4%)
LDR leaf ambient      4702/65536      112848/1572864  ( 7.2%)
HDR leaf ambient         0/65536           0/1572864  ( 0.0%)
occluders                9/0             360/0        ( 0.0%)
occluder polygons       22/0             264/0        ( 0.0%)
occluder vert ind       96/0             384/0        ( 0.0%)
detail props          [variable]           1/307712   ( 0.0%)
dtl prp lght          [variable]           1/4        (25.0%)
HDR dtl prp lght      [variable]           1/4        (25.0%)
static props          [variable]           1/30954    ( 0.0%)
pakfile               [variable]     1072116/0        ( 0.0%)

Level flags = 0

Win32 Specific Data:
physics               [variable]      702511/4194304  (16.7%)
==== Total Win32 BSP file data space used: 9266849 bytes ====

Total triangle count: 25537
Writing d:\half life 2\steamapps\dariodee\sourcesdk_content\hl2mp\mapsrc\dm_plaz
a17.bsp
13 minutes, 58 seconds elapsed
        1 file(s) copied.
Press any key to continue . . .

Funny that the list at the end says "vertexes"... as opposed to "vertices", the proper way

Edit: could the compile time have anything to do with the fact that I compile using a batch compiler? No, right?

 

[Unarmed]

Dude....

107 hours, 10 minutes, 52 seconds elapsed for VIS just screams out bad optimization. This is one way or another going to affect your gameplay. And looking at the rest of the stuff, your map cant be that huge. And even if there are huge open spaces in your map then VIS should never use this time, Open spaces are easy to VIS:

Optimize man. And then optimize again. And then agian. Keep doing that until you are below 5 hours for a full compile. 4 days just aint cutting it if you know what I mean.

 

[Dario D.]

The map is optimized nearly to death, but the layout is hopelessly open and exposed.

You can see it here: http://www.deefrag.com/hl2/p17_shots.htm

The map itself runs at an average of 50 fps for me (better than Underpass). I think the only solution to compile time is to take the deco outside the play area and turn it into a 3d skybox, but that's impossible (I tried it). I can't scale down the many crucial meshes I use, and half the beauty of the map is how the exterior deco blends with the play area.

I took a look in GL View, and the only serious problem I noticed is how Source splits every 1024 units. I don't understand the complication though, because there are maps much larger than this out there, with even more splits. Also, I realized that the Hint brushes that are supposed to shave off the rooftops only serve to double the leaf count, by splitting them... so, I removed those.

 

[Unarmed]

Hmmm...

I dont know what it is then.

But it's a gorgeous map, I can tell you that.

 

[Dario D.]

Ah. I figured it out. I removed three massive hint brushes that chopped my entire map into what were supposed to be large leafs (one that shopped off all the rooftops, one that chopped off both X-axis sides of the main, central building, and one that chopped off the two Y-axis sides - all of which extended to the far edges of the level, diving the whole map, from the ground to the sky.

From a performance standpoint, that should cut the level nicely into leaves (in fact, the map ran about 8 fps faster that way), but from a compile standpoint, the leaf count was skyrocketing, because the hints were creating more splits through EVERY existing split (since Source creates one every 1024 units, there were a TON, and they were divided further when the hints sliced through them).

So, anyway, full compile time is now down to 2 hours... from 3 days. I decided to remove the large hints, because I looked into the sample SDK maps from HL2, and realized they don't use large hints like that. There's only one instance (in the sample C17 strider map) where the entire map is divided by 2 large hint brushes that chop off the rooftops at two different altitudes.

The quantity of seemingly always-visible things in that map is astounding (it's incredibly open, and absolutely overridden with large, detailed meshes), and I have no idea how they got it to run well, unless those previously mentioned hint brushes are doing an insanely remarkable job.

 

[Unarmed]

I would keep it like that as long as the map is still WIP and then when it goes to release you add them.

Ingame FPS is worth a long compile.

 

[myth]

sheesh...thats way too long..

 

[Unarmed]

If you get 8 FPS for it, its worth it IMO.

 

[ShaolinMonk]

man i always have this problem in mapping its annoying, right now my tiny surf map takes about 3 hours to compile! and my main single player map which is pretty small takes 5+ hours!

thanks for this tutorial i will have to read over it, i hate working on my map then having to compile it overnight while i sleep

 

[myth]

thx 4 the link mate. dat was helpful...

 

[Unarmed]

huh?