Spero: More Information

[grahamperrin]

… We have two more detection/quarantine fixes to ship soon (with the upcoming release on the 26th or just after) which allow ETHOS to be used in Custom Scan and on Exec which it currently is not set up for. We're also feeding about 30 times more data into ETHOS now and it's conviction rates are trending up nicely. However, it's still a bit heavy for my liking. SPERO is expected to go live in the middle of August, we have some cool announcements around that which we'll put out around then. Also we have another engine coming in the fall (two actually). Detection efficacy and language support are our two big pushes over the next 3 months. We've a long way to go but we will get there. …

 

At http://www.clamav.net/lang/en/about/win32/ (undated) and http://vrt-sourcefire.blogspot.com/2010/08/clamav-release-announcements.html (16 August) I see that SPERO is now featured, and that it is a

 

> machine learning based protection engine

 

— but machine learning doesn't convey much, and additional information about SPERO is not easy to find.

 

I look forward to the SPERO-related announcements …

[grahamperrin]

… Ethos (heuristics) and Spero (cloud) … 

 

ETHOS is also cloud-oriented.

 

additional information about SPERO is not easy to find.

 

Sorry — whilst looking in the announcements area, and in the blog, I didn't think to look in the product:

 

 

 

 

Still, I'd like to understand more about the machine learning based models.

 

Is SPERO oriented to data from my community?

[alfred]

ETHOS is also cloud-oriented.

 

Is SPERO oriented to data from my community?

 

 

SPERO is trained on data from the whole Community versus just single users. There is generally not enough data from one user to make meaningful training set for us. SPERO uses machine learning techniques to build essentially what are decision trees for malware. Without getting into too many specifics SPERO works on a real time feedback model. You'll start seeing more SPERO detections this fall as we begin to rely more heavily on it. As you note, ETHOS is also cloud based.

 

al

[grahamperrin]

OK, I now have a clearer picture of SPERO alongside ETHOS —

 

1. Generic detection of threats through broad hashing. We look for things that look 'like' threats we know of and try to further analyze them for conviction so we can protect the community. This can also be called a 'heuristic' engine if you like. Our generic engine is ETHOS; we have another planned … SPERO.

 

… ETHOS … still a bit heavy for my liking. SPERO …

 

SPERO … decision trees for malware … 

 

— both engines are heuristic (or heuristic-like) and cloud-based

 

— SPERO is lighter than ETHOS

 

— SPERO takes a more modern decision tree approach

 

… and if I want to know more about decision trees for malware I might read e.g. Learning to detect and classify malicious executables in the wild (2006), An intelligent PE-malware detection system based on association mining (2008) or Malware Detection using Statistical Analysis of Byte-Level File Content (2009).

 

(I don't intend to read those papers. Just getting a half-mile high view of things.)

 

Without getting into too many specifics SPERO works on a real time feedback model.

 

I'm still a little confused about what, if anything, is pushed from the cloud. At http://www.immunet.com/about/index.html :

 

… pushes intelligent protection in real-time to ALL Immunet users within milliseconds.

 

… all virus detection files and intelligence for blocking viruses resides within the Immunet Cloud …

 

If all files and intelligence are in the cloud, then what is pushed to clients on the ground?

 

Thanks

Graham

[alfred]

OK, I now have a clearer picture of SPERO alongside ETHOS —

 

 

 

 

 

 

 

— both engines are heuristic (or heuristic-like) and cloud-based

 

— SPERO is lighter than ETHOS

 

— SPERO takes a more modern decision tree approach

 

… and if I want to know more about decision trees for malware I might read e.g. Learning to detect and classify malicious executables in the wild (2006), An intelligent PE-malware detection system based on association mining (2008) or Malware Detection using Statistical Analysis of Byte-Level File Content (2009).

 

(I don't intend to read those papers. Just getting a half-mile high view of things.)

 

 

 

I'm still a little confused about what, if anything, is pushed from the cloud. At http://www.immunet.com/about/index.html :

 

 

 

If all files and intelligence are in the cloud, then what is pushed to clients on the ground?

 

Thanks

Graham

 

Those papers are good reading. I am not sure I agree with their conclusions or testing methodologes but I believe the ideas are sound and are reflective of the technology we use. A decent way to think about is that the client itself and our engines on the desktop profile the files and then send that raw data to cloud where it's refined and weighed. The data back is what we call a 'Disposition' which tells the client what to make of the file. Each Disposition has actions associated with it (Ignore, Quarantine, send to the cloud, etc).

 

We try to keep the decision support 'cloud side' and have the clients simply do the intelligence gathering and then reconciliation based off the Disposition.

 

al

[grahamperrin]

… a 'Disposition' which tells the client … (Ignore, Quarantine, send to the cloud, etc).

 

The picture's much clearer now, thanks.

 

Just one question remaining re: the push, but I've taken us off-topic from the original SPERO question so it's a separate topic:

push to all clients