Nieuwe aanpak virus detectie door Norman

[Verwijderd]

Op de newsgroup alt.comp.virus heeft zich een interessante discussie afgespeeld mbt tot een nieuw concept over het detecteren van malware. Nieuwe programmaas moeten eerst hun onschuld aan tonen door ze te laten draaien in een vrituele pc. Het biedt het programma een pseudo, niet van echt te onderscheiden OS aan. Vervolgens wordt gelogged wat het programma ermee doet. Wil het e-mailen via port 25 (sntp) dan wordt een tweede virtuele machine aangemaakt die dat ontvangt en wordt ook daar gekeken wat ermee gebeurd. Dit gaat veel verder dan heuristisch scannen, je hoeft niet te decompileren om te zien wat het doet, je weet wat het doet, iedere interactie met het OS is vastgelegd.

Hieronder een deel van de discussie tussen enerzijds een vertegenwoordiger van Norman en verder [PAX], een vroegere virusschrijver lid van de groep [Slam], en Nick Fitzgerald, vroeger editor van VirusBulletin.
Frederic Bonroy opende de discussie:

Veel lees plezier,

From: Snorre Fagerland ([e-mail])
Subject: Re: Norman developing "revolutionary" sandbox?
Newsgroups: alt.comp.virus, alt.comp.anti-virus
View: Complete Thread (23 articles) | Original Format
Date: 2002-01-18 08:53:33 PST


On Wed, 09 Jan 2002 21:52:45 +0100, Frederic Bonroy <[e-mail]>
wrote:

>There is a short article about a sandbox being developed by Norman
>in a German news magazine.
>The article sounds as if the sandbox being developed was
>a revolutionary new concept. Is it? I've heard of sandboxes before...
>Does anybody have more information on this?


Certainly.

First of all - forget the word "Sandbox". This is not a sandbox in the
way the word has been used before.

Other vendors have used the word to mean "a protected area where
software may operate, but under scrutiny". In these scenarios, the
possibly suspect software is run, but it's access to system resources
is limited - f.ex. file accesses are checked to see whether they do
anything sinister. Such actions are then stopped before they can do
anything wrong. Such setups are normally platform-dependent, and you
can not be 100% sure that the tested program will not detect that it's
running in a sandbox and attempt to break out.

This is not the case here.

Our concept is really simple. We create a simulated PC inside the
scanner. The simulated PC contains simulated hardware, complete with
CPU, hard disk, screen, network devices. The hard disk has software
installed on it, and the PC actually boots like a normal PC. However,
none of the software runs on the real CPU - it runs through our CPU
emulator. The operating system components are also written by us, so
that our OS *looks like* the real OS, but since it is only pretending
to be an OS it does not have to do all the stuff a real OS does.

F.ex, if the program wants to put up a messagebox on the screen, we
only pretend to put up a messagebox, and then tell the program "OK,
done that".

After the PC is booted, the suspect file is copied into it and started
in the emulator. We just let it fly. It can do whatever it wants. It
can delete files. It can format the hard disk. It can infect files. It
can attempt to connect to other computers and send mail. Since all
these actions are performed by our CPU emulator and our version of
different operating systems, we know everything the program does. A
log file outpout from this system will show you a lot of info -
including access requests and FATchain changes.

If, f.ex. a program file is written to in our simulated PC, we halt
the system, set up a blank PC again, and copy in the file that was
changed in the first run. If that again writes to a program file, we
know it is a virus. We don't have to guess. We know.

Same thing if the program accesses the network on port 25 - we create
another blank PC, and lets it answer, but this time as mail server. If
the mail server receives an attachment that again performs the same
action, we know it is a network worm.

Now, this sounds daunting, but you won't see anything of this. You
won't have to install anything extra - it's all enclosed inside the
scan engine, so all this happens during a normal file scan.

The benefit of this is also that we can change context depending on
file format. If it is a DOS COM file, it may be enough to run DOS. If
it's a Win32 executable, we boot into Win9x or NT. If it is an ELF
executable, we boot Linux. No difference to you.

That said, of course the system has limitations. It does take up some
resources - approx. 1.3Mb per simulated PC with 4GB address space, and
it does use somewhat more time per file than a normal file scan. Also,
the system detects malicious programs based on their actions. Now, if
the program for some reason would not do anything bad, we won't alarm
- but that doesn't mean that it won't do something bad tomorrow.

We acknowledge the fact that the simulated OS inside is obviously
never going to be the real thing. There is a chance that the tested
program will detect the unnormal circumstances; it can then do a
number of things.

1. It can format the hard disk. Stupid, because we would then just
recognise a trojan.

2. It can attempt to break out of the sandbox. Sorry, no can do. There
is nothing to break out of, so to speak - the program is run by OUR
emulator, and if the program tries to put the emulator in a situation
it does not know how to handle, it just stops emulating.

3. It can avoid doing bad things. This is probably the worst scenario
- because we then won't detect anything. This situation would however
be quickly fixed by us in a definition file update, where we have
updated the simulated OS to perform in a more "lifelike" manner.

Message 10 in thread
From: Snorre Fagerland ([e-mail])
Subject: Re: Norman developing "revolutionary" sandbox?
Newsgroups: alt.comp.virus, alt.comp.anti-virus
Date: 2002-01-18 17:06:19 PST


On Fri, 18 Jan 2002 18:30:13 GMT, "Lee Higdon"
<[e-mail]> wrote:

>
>"Snorre Fagerland" <[e-mail]> wrote in message
>news:[e-mail]...
>
>> 3. It can avoid doing bad things. This is probably the worst scenario
>> - because we then won't detect anything. This situation would however
>> be quickly fixed by us in a definition file update, where we have
>> updated the simulated OS to perform in a more "lifelike" manner.
>>
>>
>Under condition 3., if this were to happen to in userland, and not in a
>controlled environment, how would you know a definition file update is
>needed? How would the user know? I'm not understanding this scenario.


Because, just as it is today, users will once in a while become
infected. Viruses will come that are not detected by the NVC sandbox
or other more traditional tecniques used by NVC. They might be found
by other antivirus programs, or they may not. They may be found by the
users themselves. When this happens, the users come screaming to us.

In cases where the sandbox does not find anything, we'll sit down,
figure out why and make the fix. Sometimes we may have to say "Ok,
this specific type of malware can not be detected using this approach"
and we'll detect them using the ordinary string/crc scanning or
regular heuristics we use today.

From: [PaX] ([e-mail])
Subject: Re: Norman developing "revolutionary" sandbox?
Newsgroups: alt.comp.virus, alt.comp.anti-virus
View this article only
Date: 2002-01-18 12:06:04 PST


>>>We acknowledge the fact that the simulated OS inside is obviously
never going to be the real thing. There is a chance that the tested
program will detect the unnormal circumstances; it can then do a
number of things.
<<<

Ok sounds good in principle...anti emulation techniques have been around for
some time though,how do you pass control back to the *OS* prime??
Sounds a little like a resource hogger?
What sort of CPU time is it using?
Will it see Format.exe as a trojan?
Whats to stop it being unloaded from memory using normal
techniques?(BumbleBee did a very good tutrorial for unloading AV programs).
How does it cope with such things as Z0mbies Zmist?

The words "Revolutionary" have been used before(by TBav to name just one and
look what happened to them)

rgrds Dalt

From: Nick FitzGerald ([e-mail])
Subject: Re: Norman developing "revolutionary" sandbox?
Newsgroups: alt.comp.virus, alt.comp.anti-virus
View: Complete Thread (23 articles) | Original Format
Date: 2002-01-18 13:36:06 PST


"[PaX]" <[e-mail]> wrote:

Obviously I can't answer with the details, but for a clever chap you
asked some fairly naive questions there Dalt...

> Ok sounds good in principle...anti emulation techniques have been around for
> some time though,how do you pass control back to the *OS* prime??

As the emulator (if properly written) is the only machanism that can
handle the nearest thing that their engine does to "passing control
back to the OS", the answer does not matter. Remember, this is *not*
the old "run safe operations on the real CPU" type of "emulation".
Because that is only psuedo-emulation, it is very easily tricked, as
many of the anti-emulation techniques you refer to show.

Such techniques, however, are academic in a thorough-going emulator.
What you need to do to"anti-emulate" this approach is a reliable way
for your code to detect that it is running in the NVC emulator and
then not do anything bad. Of course, the trouble with that is that
if there is such a mechanism (and there could well be many) when the
code that implements it is found through the other mechanisms that
malware is found all the Norman techies have to do is add detection
for that type of code to the emulator *or* "improve" the emulator
"shortcoming" that the detection technique works off.

> Sounds a little like a resource hogger?
> What sort of CPU time is it using?

Don't know -- not seen it performance profiled...

However, I think the surprising thing about this approach is it shows
how coparatively small the overhead really is nowadays. This is bound
to have something to do with "normal" CPU loads now being well below
10%...

> Will it see Format.exe as a trojan?

As it has to emulate the existence of FORMAT.EXE, I guess it should
"know" what FORMAT.EXE is supposed to do. Of course, there are
"intentionality issues" with determining that *anything* is Trojan,
but as these apply equally for human and machine-based analysis I
don't see this example being any more or less problematic than for
any other programmatic approach...

> Whats to stop it being unloaded from memory using normal
> techniques?(BumbleBee did a very good tutrorial for unloading AV programs).

Did you not read Snorre's description? There is no such thing as
"normal techniques" for unloading the emulator **from inside the
emulator** because none of the emulated code is running on the real
CPU. If someone writes something new that reliably detects it is
running in the emulator and then falls into a "be nice" mode so the
emulator does not detect anything "bad", that code may then get to
be run and *then* may try whatever it likes to unload the on-access
scanner however they will. That's fine, because it gets around the
"how do we trick the emulator" issue, above. However, a few users
will quickly notice that their scanner is being disabled when
certain programs are run, samples will be sent to Norman for analysis
and the emulator "fixed" to circumvent the first issue.

Don't forget -- NVC does not rely **only** on this fancy emulator
for its malcode detection. Layered approaches are still better than
employing a single technology.

> How does it cope with such things as Z0mbies Zmist?

Specifically, I don't know. It should detect it just fine from the
recursive "modifies other EXE" detection mechanism though.

> The words "Revolutionary" have been used before(by TBav to name just one ...

Indeed!

> ... and
> look what happened to them)

Yep -- they were bought by Norman...


--
Nick FitzGerald

On Fri, 18 Jan 2002 20:05:35 +0000 (UTC), "[PaX]" <[e-mail]>
wrote:

>>>>We acknowledge the fact that the simulated OS inside is obviously
>never going to be the real thing. There is a chance that the tested
>program will detect the unnormal circumstances; it can then do a
>number of things.
><<<
>
>Ok sounds good in principle...anti emulation techniques have been around for
>some time though,how do you pass control back to the *OS* prime??

It could attempt to crash the emulator. Won't work. All legal CPU
instructions including MMX and FPU are emulated. Illegal instructions
fall into the "unknown" category and then the emulator just stops.

It could attempt to fool the emulator by running innocent code instead
of malicious code. Could work, I'm sure there are tons of ways to
detect that the code is running under an emulated OS, but then the bad
guys had to come up with new ways to detect this every time we fix
such a loophole.

It could come up with some unbeatable trick to fool emulators.
Possible, but we still have our non-emulating methods to detect
viruses.

I'm not sure about what you mean with OS prime. This happens inside
the scanner - the emulation of the file continues until one of a few
stop conditions are met, which I won't go into details on here.


>Sounds a little like a resource hogger?
Yep it does. It uses more time than regular scan. However, we are
researching ways to shorten the time spent on each file. F.ex. the
boot process of the simulated PC is normally done only once. After
this, snapshots are made of the different machines, so it is in
practice only a memcpy to set up a new blank one. Files are also run
in the simulated PC using a technique called "Page fault - opcode
restart" which basically means that the file is loaded as execution
flows into areas which have not been loaded yet. This means we don't
have to load the entire file.

>What sort of CPU time is it using?
Varies a lot, but the engine does yield, so even if it would long time
on a file, the rest of the machine keeps on operating normally.

>Will it see Format.exe as a trojan?
Nope, but you are touching an important issue we will have a hard look
at - how to handle malicious actions hidden behind a wall of user
interaction.

>Whats to stop it being unloaded from memory using normal
>techniques?(BumbleBee did a very good tutrorial for unloading AV programs).
This is implemented in the scan engine, so if you can remove the
scanner you can remove this too. However, AV unloading has been around
since old DOS days and has not, I must say, been a very successful
approach seen from the virus point of view. Deleting/disabling the AV
installation is a surefire way of getting the virus noticed fast.

>How does it cope with such things as Z0mbies Zmist?
ZMist is a special case. However, what causes most problems for the
emulator is not the fact that ZMist is extremely metamorphic (in fact
this part of the engine was developed partly to handle metamorphism)
but instead the EPO techniques employed by this virus. EPO (Entry
Point Obscuring) causes the engine to have to sometimes chew through a
lot of totally innocent code. So, whether it detects ZMist or not
depends actually a lot on what the host program does before the virus
gets control. But as mentioned before, special/difficult cases can
still be detected using other techniques.

>
>The words "Revolutionary" have been used before(by TBav to name just one and
>look what happened to them)

Yep, they were bought by us
But hey, I think the method is fairly cool, even if I wasn't the evil
mastermind behind the marketing ploy.

De hele thread is te vinden op
http://groups.google.com/groups?hl=en&threadm=u4kol7r7h8vb43%40corp.supernews.com&prev=/groups%3Fgroup%3Dalt.comp.virus

[Gerco]

Ik moet zeggen: "Dat klinkt erg goed"

Maar misschien ben ik iets te voorbarig als ik zeg dat het erg simpel is om die emulator te omzeilen. Kijk maar naar het voorbeeld van de messagebox: Als ik als prog 10 messageboxjes met OK op het scherm zet en die worden binnen een bepaalde tijd (zeg 1 sec) allemaal gesloten, dat ben je BIJNA zeker dat er geen user achter die PC zit, maar de emulator.

Nu is dat voorbeeld van die messagebox niet zo geslaagd voor een virus aangezien die "stiekum" moeten zijn, maar het is ongetwijfeld mogelijk om op andere manieren te zien dat er user het programma niet runt, zeker voor trojans.

[Aaargh!]

Ik vind het een beetje vaag klinken, als ze dit correct willen laten werken moeten ze de hele win32 api implementeren binnen hun gemuleerde PC/OS.
de Wine mensen zijn daar al jaren mee bezig en het is nog lang niet optimaal, en zij willen dit zo ff uit hun mouw schudden

[Gerco]

Op maandag 21 januari 2002 15:42 schreef Aaargh! het volgende:
Ik vind het een beetje vaag klinken, als ze dit correct willen laten werken moeten ze de hele win32 api implementeren binnen hun gemuleerde PC/OS.

Dat is dus niet helemaal het geval. Ze zullen een redelijk deel van de Win32 API moeten EMULEREN!. Dingen als disk-writes moeten inderdaad ook echt geschreven worden. Maar alles wat met graphics te maken heeft kun je afdoen met een "return 0;" en da's toch een behoorlijk groot deel van windows.

Dat betekent niet dat het nog steeds een ENORME klus is om zoiets te maken en ik betwijfel of het ooit helemaal afkomt, maar het IDEE is wel goed.

[Verwijderd]

Ik denk dat dingen als diskwrites gefaked worden inclusief checkes en zeker schrijven in de registry.
Het moet een volledig afgeschermd geheel zijn waar een virus zodanig afgesloten is van het systeem, dat het daar geen enkel kwaad kan, maar alleen laat zien wat het probeerd te doen.

In wezen bestond dit al, maar dan als behoorlijk dure VM programma's en virus experts werkten er al mee.

[koekeeghan]

Fascinerend idee die sandbox. Helaas lees ik weinig over de resources die het gebruikt. Ik heb in een tijdschrift gelezen dat ie sneller dan Norton zou scannen ondanks de dubbele scan methode.

[momania]

Ook oude

Nogal nutteloze toevoeging