Efficacy Testers Needed

[alfred]

All,

 

I am in the process of testing some rather aggressive new detection regimes in both our SPERO and ETHOS engines. If you have a VM and are willing to help test for True and False positives, your welcome to come on board. Mail me at alfred@immunet.com.

 

The caveat here is that this is early in the testing cycle and the goal is to find FP and TP rates both so unpleasant things are likely to happen to the test environment you are using, hence the requirement for a VM is a must.

 

Best,

Alfred

[Guest Orlando]

All,

 

I am in the process of testing some rather aggressive new detection regimes in both our SPERO and ETHOS engines. If you have a VM and are willing to help test for True and False positives, your welcome to come on board. Mail me at alfred@immunet.com.

 

The caveat here is that this is early in the testing cycle and the goal is to find FP and TP rates both so unpleasant things are likely to happen to the test environment you are using, hence the requirement for a VM is a must.

 

Best,

Alfred

 

You can count on my help. I'm currently playing Immunet with some niche antivirus (now Zillya!). Although the tests are not desired, I will send the problems encountered with other antivirus software (if any). I will send some false positives, etc..

 

Regards,

Orlando

[alfred]

You can count on my help. I'm currently playing Immunet with some niche antivirus (now Zilly!). Although the tests are not desired, I will send the problems encountered with other antivirus software (if any). I will send some false positives, etc..

 

Regards,

Orlando

 

Orlando,

 

Do you have a virtual machine set up?

 

al

[Guest Orlando]

Orlando,

 

Do you have a virtual machine set up?

 

al

 

In my personal computer I haven't the virtual machine, but in the test computer there is the virtual machine to do any testing. Currently, however, has merged the CPU (computer test). The computer with the virtual machine will be operational by Monday.

 

Regards,

Orlando

[alfred]

In my personal computer I haven't the virtual machine, but in the test computer there is the virtual machine to do any testing. Currently, however, has merged the CPU (computer test). The computer with the virtual machine will be operational by Monday.

 

Regards,

Orlando

 

OK, Great, just let me know.

[alfred]

All,

 

I am in the process of testing some rather aggressive new detection regimes in both our SPERO and ETHOS engines. If you have a VM and are willing to help test for True and False positives, your welcome to come on board. Mail me at alfred@immunet.com.

 

The caveat here is that this is early in the testing cycle and the goal is to find FP and TP rates both so unpleasant things are likely to happen to the test environment you are using, hence the requirement for a VM is a must.

 

Best,

Alfred

 

So far I am testing the Generic detect tree (lovingly named W32.SPERO.Vacuum.P1) and the results are very promising.

 

If you are interested in testing - here is a quick cheat sheet:

 

 

 

1) Install new version from http://www.immunet.com/free .

 

2) Open cmd window as Administrator.

 

3) Stop the immunet protect service "sc stop immunetprotect"

 

4) From the same cmd window run ' notepad "c:\Program Files\Immunet

Protect\1.0.22\global.xml" '

 

5) To the node add the following sub-nodes

 

b.immunet.com

cloud14.immunet.com

 

Typically I add them right before the tag.

 

 5b) Find the agent\submission\http_upload_server key and set it's value to cloud14.immunet.com

 

5c) Save global.xml

 

6) Next delete the local.xml file from the immunetprotect root directory.

 

7) Run agent -r from the version directory.

 

8) Run agent -r from the version directory (Because of a bug)

 

9) Restart the agent service "sc start immunetprotect"

 

When you are introducing malware to your system, if it's in bulk you will want to shut off IMP to get it onto the system to be scanned. You can use the sc stop immunetprotect command for this and the sc start command to get it going again.

 

For this part of the process we are testing a Generic sub-engine and a Family specific engine. The two detection names you are looking to see are:

 

1. W32.SPERO.Vacuum.P1

2. W32.SPERO.Allaple

 

Each test should generally be done in this order when testing for SPERO

 

1. Bring malware set on (with IMP disabled), fire up IMP and set SPERO in the settings to OFF.

 

2. Scan and note your conviction count.

 

3. Turn of IMP and issue this command:

 

c:\ del c:\program files\immunet protect\cache.db

 

4. Move the same test set back onto the system and fire up IMP again.

 

5. Enable SPERO and scan. Note the results.

 

Here is a basic example of what my simple testing looks like:

 

Test Set 1

 

1138 pieces of Malware

IMP Without SPERO - 586 Convictions

IMP With SPERO 971 Convictions

 

As for clean files, this is fairly simple as well. Install your favorite programs and see if we end up with any false detections. If so, please let me know what they are and where I might find them.

[Guest Orlando]

So far I am testing the Generic detect tree (lovingly named W32.SPERO.Vacuum.P1) and the results are very promising.

 

If you are interested in testing - here is a quick cheat sheet:

 

 

 

1) Install new version from http://www.immunet.com/free .

 

2) Open cmd window as Administrator.

 

3) Stop the immunet protect service "sc stop immunetprotect"

 

4) From the same cmd window run ' notepad "c:\Program Files\Immunet

Protect\1.0.22\global.xml" '

 

5) To the <cloud> node add the following sub-nodes

 

<server>

b.immunet.com

</server>

<host>

cloud14.immunet.com

</host>

 

Typically I add them right before the </cloud> tag.

 

 5b) Find the agent\submission\http_upload_server key and set it's value to cloud14.immunet.com

 

5c) Save global.xml

 

6) Next delete the local.xml file from the immunetprotect root directory.

 

7) Run agent -r from the version directory.

 

8) Run agent -r from the version directory (Because of a bug)

 

9) Restart the agent service "sc start immunetprotect"

 

When you are introducing malware to your system, if it's in bulk you will want to shut off IMP to get it onto the system to be scanned. You can use the sc stop immunetprotect command for this and the sc start command to get it going again.

 

For this part of the process we are testing a Generic sub-engine and a Family specific engine. The two detection names you are looking to see are:

 

1. W32.SPERO.Vacuum.P1

2. W32.SPERO.Allaple

 

Each test should generally be done in this order when testing for SPERO

 

1. Bring malware set on (with IMP disabled), fire up IMP and set SPERO in the settings to OFF.

 

2. Scan and note your conviction count.

 

3. Turn of IMP and issue this command:

 

c:\ del c:\program files\immunet protect\cache.db

 

4. Move the same test set back onto the system and fire up IMP again.

 

5. Enable SPERO and scan. Note the results.

 

Here is a basic example of what my simple testing looks like:

 

Test Set 1

 

1138 pieces of Malware

IMP Without SPERO - 586 Convictions

IMP With SPERO 971 Convictions

 

As for clean files, this is fairly simple as well. Install your favorite programs and see if we end up with any false detections. If so, please let me know what they are and where I might find them.

 

Bad news for me, the CPU will come in a week (due to late shipment), the coputer test will be operational next two weeks. I'll inform you when arrive.

 

Regards,

Orlando

[christhomas]

I guess, I will test this sometime later when there won't be need for a virtual environment. But it sure reads promising

[Shaoran]

Hi,

 

For me it was not "c:\Program Files\Immunet Protect\1.0.22\global.xml" but "c:\Program Files\Immunet Protect\2.0.15\global.xml"

 

So, I try it with 4 malware folders :

 

2010 : 2903 threats

2009 : 1068 threats

2008 : 297 threats

< 2005 : 5000 threats

 

without update

2010 : 1292 threats not detected

2009 : 5 threats not detected

2008 : 126 threats not detected

< 2005 : 2520 threats not detected

 

 

with update

2010 : 813 threats not detected

2009 : 1 threats not detected

2008 : 124 threats not detected

< 2005 : 2518 threats not detected

 

 

So yes, it seems to find more threats, but immunet has still one of the worst score

[alfred]

Hi,

 

For me it was not "c:\Program Files\Immunet Protect\1.0.22\global.xml" but "c:\Program Files\Immunet Protect\2.0.15\global.xml"

 

So, I try it with 4 malware folders :

 

2010 : 2903 threats

2009 : 1068 threats

2008 : 297 threats

 

without update

2010 : 1292 threats not detected

2009 : 5 threats not detected

2008 : 126 threats not detected

 

 

with update

2010 : 813 threats not detected

2009 : 1 threats not detected

2008 : 124 threats not detected

 

 

So yes, it seems to find more threats, but immunet has still one of the worst score

 

 

You need to keep in mind this is a small set of cloud sub engines, we can have dozens. No single one will be a panacea. Also, we will do better on newer malware because we focus on it. Malware from 2008 etc. is not something you should expect our product to detect well, it's also not something people will see in the field with any real frequency. The results on the 2010 and 2009 folders make me very happy.

 

Thanks a bunch for working with us on this.

 

al

[Shaoran]

I should precise about the worst score, I was speaking about the first pack (2010) wich is my own pack unlike the others. If you take some free av like avast, antivir, panda cloud or avg, they left something between 100 and 250.

 

But, for sure, Immunet is still young, it just need to grow

[alfred]

I should precise about the worst score, I was speaking about the first pack (2010) wich is my own pack unlike the others. If you take some free av like avast, antivir, panda cloud or avg, they left something between 100 and 250.

 

But, for sure, Immunet is still young, it just need to grow

 

 

Of course, this is one engine. We are releasing over a dozen in the next two months. This field test is exactly that, just a test. So far the results look fantastic.

 

al

[buckslayr]

Of course, this is one engine. We are releasing over a dozen in the next two months. This field test is exactly that, just a test. So far the results look fantastic.

 

al

 

Hi Al,

Just a quick question regarding the new engines. At some point in the future with the new engines in place, will there still be a need for the plus version or will the cloud only version give adequate protection as a stand alone?

[Shaoran]

Second part of the test, scan of safe files (All my data), there is 5 new false positif, one on BleachBit, one on AdslTV, two on Company of Heroes, one on Dawn of War - Soulstorm (cracked dll only (I hate to insert my cd to play ^^) original file hasn't been detected)

 

 

If you want this files, just tell me where upload them

[Guest Orlando]

Second part of the test, scan of safe files (All my data), there is 5 new false positif, one on BleachBit, one on AdslTV, two on Company of Heroes, one on Dawn of War - Soulstorm (cracked dll only (I hate to insert my cd to play ^^) original file hasn't been detected)

 

 

If you want this files, just tell me where upload them

 

You can report false positives in this page: http://www.immunet.com/contact/index.html in the dropdown choose "submit a false positive".

 

Thanks for the support,

Orlando

[Shaoran]

Hi,

 

Is this update already added in Immunet without the modifications ?

[alfred]

Hi,

 

Is this update already added in Immunet without the modifications ?

 

 

Not sure I understand the question.

[alfred]

Second part of the test, scan of safe files (All my data), there is 5 new false positif, one on BleachBit, one on AdslTV, two on Company of Heroes, one on Dawn of War - Soulstorm (cracked dll only (I hate to insert my cd to play ^^) original file hasn't been detected)

 

 

If you want this files, just tell me where upload them

 

 

The easiest way to do this is actually to just roll them back out of quarantine. I get all of those logs and will examine the data there. Sending them (as Orlando suggested) is also quite helpful.

 

al

[Shaoran]

Fp have already been fixed. What I means is if you have already sent this update to all Immunet client.

I ask you that because I ran some test recently, and I see that Immunet have a better detection, and there is not any new detection with this test.

[Shaoran]

In fact, I found the answer in another thread, I didn't know that SPERO send detected files to the cloud.

[Blueberries]

we will do better on newer malware because we focus on it. Malware from 2008 etc. is not something you should expect our product to detect well, it's also not something people will see in the field with any real frequency. The results on the 2010 and 2009 folders make me very happy.