BCM Episode2 InstallShield Installer Bug ?

Page 1 of 1

You cannot start a new topic
You cannot reply to this topic

BCM Episode2 InstallShield Installer Bug ?

#1 Guest_zee_*

Group: Guests

Posted 16 August 2001 - 07:58 PM

The InstallShield .msi wrapper which was used to package the BCM E2, seems to have a quite serious free space detection bug.
(Yes, I understand that it is not related in any way to the BCM itself).

***WARNING*** BEGIN_TECHNICAL_NONSENSE

The wrapper performs these operations:
1. Check for presence of Microsoft Installer Runtime. If it's not installed, or if it's older than the version packaged inside the wrapper, then the wrapper installs the MSI Runtime, and reboots the system. Also, it registers the wrapper to be run again when the system reboots (via the RunOnce key).

(The wrapper .EXE has the MSI runtime distribution inside in a standard CAB archive)

2. The wrapper then searches for a temporary place capable of storing a .MSI package contained within the wrapper. This is exactly the place where integer overflow problems occur.

3. The wrapper extracts the .MSI package to chosen temporary location. Oddly, it does this multi-megabyte copying that can take several minutes completely in the background, without any possibility to cancel it except killing via the Task Manager.

4. After the .MSI is unpacked, the 'msiexec /i <.MSI file name>' (or similar command, I didn't test if it runs exactly this command) is launched.

5. The wrapper justs hangs until the msiexec process quits (probably with WaitForSingleObject or similar API call). After the msiexec quits, it removes all temporary files created, and exits.

6. The .MSI package itself works well (it copies the .MSI file to the temporary directory yet another time, but disk space detection is correct).

-----
Now, about the Nasty Bug that crippled into the Wrapper from the era of 60 mb hard drives

***WARNING*** BEGIN_EVEN_MORE_TECHNICAL_NONSENSE

The problem is that they use old GetDiskFreeSpace API for checking free space.

There is an ancient API 'GetDiskFreeSpace'. It returns only lower 32 (or sometimes 31) bits of free disk space. So, if you have exactly 4 GB free (that is, 4294967296 bytes), it will report as if you were having 0 bytes free on that drive (at least, that is the way it behaves under Win2K). If you have 8.01 GB free, for example, then the free space reported with this API will be only 0.01 GB.
(According to Windoze documentation, it's even wrapped to 2GB, not to 4GB.)
This is a quote from a MSDN page for this API:

quote:

The GetDiskFreeSpace function cannot report volume sizes that are greater than 2 GB.
To ensure that your application works with large capacity hard drives, use the GetDiskFreeSpaceEx function.
The GetDiskFreeSpaceEx function is available beginning with Windows 95 OEM Service Release 2 (OSR2), and you should use it whenever possible. The GetDiskFreeSpaceEx function returns correct values for all volumes, including those that are larger than 2 gigabytes.

How they (InstallShield) should have done it right (IMHO): since the GetDiskFreeSpaceEx is not available on ancient pre-OSR2 systems, directly importing GetDiskFreeSpaceEx would cause DLL link resolve errors when starting on pre-OSR2 win95. They should have tried to get the GetDiskFreeSpaceEx address via a GetProcAddress(hKernelDll,"GetDiskFreeSpaceExA") call, an if it fails, use the ancient GetDiskFreeSpaceA function (that's not a problem, since old Windoze didn't support large partitions anyway).

END_EVEN_MORE_TECHNICAL_NONSENSE

Getting the bug on your system:

Now, several words about how the Wrapper searches for a location. First, it will check (mis-check ) if your TEMP directory is capable of holding the .MSI package. If it succeeds, it will unpack the MSI for example, to "C:Documents and Settingsv0rtexLocal SettingsTemp_is2EBattlecruiser Millennium Demo E2.msi". If it thinks that your TEMP directory lacks free space, it will try to use other local drives. For example, I deliberately left 4.01 GB on my C: drive and it unpacked the .MSI file to "D:_is32Battlecruiser Millennium Demo E2.msi". But i am a bad boy and for the next run I left on the D: drive only 4.01 GB
Then I got this nice screen (it didn't even have text on a button):

OS used: Windows 2000 Pro (russian).

PS: I wonder why this tiny buggy MSI wrapper is even called InstallShield, the well-recognized brand name. The MSI technology is free (available for download from MS), why do they resell it under another name? Hmmm...i'm disappointed about it... In fact, I can write a better wrapper in a day or two (just some bragging )

Please forgive me for loads of USELESS technical NONSENSE, i just enjoy writing this crap
I think that this bug is already known, but I couldn't find an exact description of it on the board.

And of course,
Long Live BC !
Long Live the SC !

#2 Charles Lindsey

Chavik

Group: Moderators
Posts: 2,846
Joined: --
Gender:Male
Location:Birmingham, Al
Interests:PC games, UC; Thief II was hard, never finished, reading, cooking, cars

Posted 16 August 2001 - 09:09 PM

And.. mare zee dotes n dozy dotes n luddle lamsy divy.

Very intersting stuff, zee.

#3 Cmdr Nova

LT. Commander

Group: Members
Posts: 1,524
Joined: 19-March 01
Gender:Male
Location:Spokane, WA (-8 GMT)

Posted 16 August 2001 - 09:20 PM

Pretty spiffy!

Welcome aboard, Zee!

#4 Supreme Cmdr

Supreme Cmdr

Group: Administrators
Posts: 16,472
Joined: 06-October 98
Gender:Male
Location:South Florida
Interests:Keeping commanders in line!

Posted 18 August 2001 - 03:43 PM

Pretty spiffy sleuthing there indeed, Zee.

Up to last week, I was using ISX 3.03. I just paid (last week), $164 for an upgrade to ISX 3.5 - so I hope the b*astards have fixed it, cuz I'd sure like to think I've paid $164 for a fix.

Anyway, just to play it safe, I'm gonna stuff it under SoftIce when I get the chance, and see if in fact they've fixed it.

I'll be packaging the next BCM demo (due out just around the game's release in Sept) with ISX 3.5 too, btw.

#5 Kush

Ensign

Group: Members
Posts: 428
Joined: 05-August 00
Location:Toronto, Ontario, Canada

Posted 19 August 2001 - 07:31 AM

Cool explanation ZEE!!!!!!

I was wondering what was up with the install shield.

[ 08-19-2001: Message edited by: Kush ]

#6 Guest_zee_*

Group: Guests

Posted 19 August 2001 - 06:05 PM

EDITED: ****WARNING****: this is again incorrect . It has the bug even on win9x! See the next posts.
END_EDITED

I also found that this bug WILL NOT affect Win9x users.
If you try to get this bug on Win9x/WinME, you will not get it.

My previous message was slightly incorrect:

1) Unlike GetDiskFreeSpaceEx, GetDiskFreeSpace returns not the size itself, but 4 32-bit numbers:
number of sectors per cluster, bytes per sector, number of free clusters and total number of clusters.

To get the free space, you must multiply these numbers:
FreeSpace = NumberOfFreeClusters * SectorsPerCluster * BytesPerSector.

Because this involves three 32-bit numbers, this multiplication must be done in 64-bit ariphmetic. Some applications still use 32-bit ariphmetic for this.
For compatibility reasons, Win9x family will return fake info about disk layout, so the total number of bytes will be almost 2 GB. ( 2GB, not 4, because if someone stores the result to a signed integer, it can become negative).
Win9x DOES NOT return low 31 bits of disk space, it will ALWAYS return almost 2GB. (my previous message was incorrect on this)

But in NT/Win2000 family GetDiskFreeSpace returns true information.
The multiplication can result in value much greater than 4GB. Hence, the multiplication must be done in 64-bits, or the result will be CLIPPED to lower 32 bits.

There is an excellent Knowledge Base article that describes how the GetDiskFreeSpace really works on different OS. It somewhat differs from MSDN GetDiskFreeSpace description.

This is a quote from it:

quote:

Windows 95/98-specific Behavior
The information in this section applies to all versions of Windows 95 and Windows 98. GetDiskFreeSpace returns a maximum total size and maximum free size of 2GB. For example, if you have a 6GB volume and 5GB are free, GetDiskFreeSpace reports that the drive's total size is 2GB and 2GB are free. This limitation originated because the first version of Windows 95 only supportsed volumes of up to 2GB in size. Windows 95 OSR2 and later versions, including Windows 98, support volumes larger than 2GB. GetDiskFreeSpaceEx does not have a 2GB limitation, because it is preferred over GetDiskFreeSpace. GetDiskFreeSpace returns a maximum of 65536 for the numbers of total clusters and free clusters to maintain backward compatibility with the first version of Windows 95. The first version of Windows 95 supports only the FAT16 file system, which has a maximum of 65536 clusters. If a FAT32 volume has more than 65536 clusters, the number of clusters are reported as 65536 and the number of sectors per cluster are adjusted so that the size of volumes smaller than 2GB may be calculated correctly. What this means is that you should not use GetDiskFreeSpace to return the true geometry information for FAT32 volumes.

Windows NT/2000-specific Behavior
On Windows NT and Windows 2000, GetDiskFreeSpace is not limited to 2GB; it reports the full size of the drive and the full amount of free space. Windows 2000 supports disk quotas. When quota hard limits are being enforced, GetDiskFreeSpace returns the quota size for the user as the total disk space and returns the user's remaining unused quota for the free space. This is done so that programs can determine how much disk space they can actually use. GetDiskFreeSpaceEx returns values subject to enforced quota limits also, but specifically returns the number of bytes available to the user who is running the program.

So, the problem is even not because they used GetDiskFreeSpace, but because they handled the results incorrectly (they assumed that GetDiskFreeSpace will always indicate < 2 GB).

[ 08-20-2001: Message edited by: zee ]

#7 Joel Schultz

Lieutenant

Group: Moderators
Posts: 563
Joined: 24-March 00
Gender:Male
Location:Penn Hills, PA, USA
Interests:UC, ping-pong, pinball, & *stuff* (tm)

Posted 20 August 2001 - 10:06 AM

Great work zee! Nice to see another developer around these parts...

#8 Guest_zee_*

Group: Guests

Posted 20 August 2001 - 07:14 PM

Aaaarghhh! That sucks! They (IShield) fooled me again !

I hope this will be the last correction.

I thought they don't call GetDiskFreeSpaceEx but in fact they DO !!!
And they've managed to interpret the result incorrectly.

They call GetProcAddress to get the GetDiskFreeSpaceEx pointer. (that's good).
When I wrote first two posts I was lazy enough to launch the debugger and check everything myself. I assumed that if it runs incorrectly, they use old API (and this API is listed in the import table). I just couldn't imagine that one can use GetDiskFreeSpaceEx and still get wrong results. But, if anything can go wrong, it will.

That means it WILL have the bug on Win9x.
I'm sorry for changing my mind twice, they've just fooled me !

The next post contains the disassembly of the faulty code with some comments (for hackers only!). Sorry again for posting such crap here.

#9 Guest_zee_*

Group: Guests

Posted 20 August 2001 - 07:17 PM

code:



....

004064F2   lea         ecx,[esp+20h]

; the 64-bit free disk space will be stored at the address, pointed now by ecx (or by esp+20h), Intel byte order

004064F6   lea         edx,[esp+30h]

004064FA   push        ecx

004064FB   lea         eax,[esp+2Ch]

004064FF   push        edx

00406500   lea         ecx,[esp+440h]

00406507   push        eax

00406508   push        ecx

00406509   call        ebp ; this is the call to GetDiskFreeSpaceEx ! (it has 4 parameters)

; esp is now the same as at 004064F2; 64-bit free space is now at esp+20h, that is:

; the low 32 bits are at dword ptr [esp+20h]

; the high 32 bits are at dword ptr [esp+24h]
0040650B   test        eax,eax ; check if GetDiskFreeSpaceEx failed

0040650D   je          004065A0 ; if it failed (that is unlikely),

; call FreeLibrary and return zero (not shown here).

; I won't discuss this because the failure is unlikely.
00406513   mov         edx,dword ptr [esp+20h] ; the low 32 bits of free disk space are now in edx

00406517   push        esi ; esi is the HINSTANCE of "kernel32.dll"

00406518   shr         edx,0Ah ; divide edx (low 32 bits of free space) by decimal 1024
; edx now contains number of free kilobytes on disk,

; but obtained only from low 32 bits of disk space
0040651B   mov         dword ptr [ebx],edx ; store that number of kilobytes

; that ebx was set at the beginning of the function (not shown here)

; The function was probably prototyped as:

; BOOL __cdecl GetFreeKilobytesOnPath( char *path, unsigned long *kbytes )

; In fact, 'path' was set to "C:DOCUME~1v0rtexLOCALS~1Temp",

; and 'kbytes' was a pointer to store the result. Now ebx = 'kbytes' pointer.
0040651D   mov         edi,1 ; indicate success

00406522   call        dword ptr ds:[40F080h] ; call FreeLibrary

00406528   mov         eax,edi ; indicate success (continued from 40651D)
; restore registers and the stack

0040652A   pop         ebp

0040652B   pop         edi

0040652C   pop         esi

0040652D   pop         ebx

0040652E   add         esp,828h

; Oops, we didn't do anything with high 32 bits of free space! We just threw them away!

00406534   ret ; return 
====================================
Now, let's see what's going on after it returns:
===================================

00406457   call        00406480 ; <-- we were inside

; <--- we are now here after returning from previous function
0040645C   add         esp,8 ; restore the stack

0040645F   test        eax,eax ; check if it succeeded

00406461   je          00406475 ; if not, goto _error
; [esp+4] contains the number of free kilobytes (obtained incorrectly)

; [esp+8] contains the number of kilobytes required for the setup

; (in this setup, that is 124236 kilobytes, or 127217664 bytes)
00406463   mov         ecx,dword ptr [esp+4]

00406467   mov         eax,dword ptr [esp+8]

0040646B   cmp         ecx,eax

0040646D   jbe         00406475

; that was pretty obvious: if FreeKilobytes <= RequiredKilobytes goto _error

; FreeKilobytes obtained incorrectly !
0040646F   mov         eax,1 ; indicate success

00406474   ret
_error:

00406475   xor         eax,eax ; indicate failure

00406477   ret