<div dir="ltr">Hmm, I just had a look and they are indeed very different. I don't have time to debug this right now, but my guess is that the record-from-snapshot feature isn't quite working correctly since we ported it to PANDA 2.<div><br></div><div>A simpler solution may just be to use a smarter binary diff program, such as bsdiff. On two panda2 snapshots created from the same initial QCOW snapshot, I got this:</div><div><br></div><div><div>moyix@lorenzo:~/git/panda/build$ ls -ld test[12]-rr-snp</div><div>-rw-rw---- 1 moyix moyix 84462802 Nov 6 10:42 test1-rr-snp</div><div>-rw-rw---- 1 moyix moyix 84454612 Nov 6 10:42 test2-rr-snp</div><div>moyix@lorenzo:~/git/panda/build$ bsdiff test1-rr-snp test2-rr-snp test2.patch</div><div>moyix@lorenzo:~/git/panda/build$ ls -l test2.patch</div><div>-rw-rw-r-- 1 moyix moyix 228 Nov 6 10:53 test2.patch</div><div>moyix@lorenzo:~/git/panda/build$ bspatch test1-rr-snp test2new-rr-snp test2.patch </div><div>moyix@lorenzo:~/git/panda/build$ md5sum test2-rr-snp test2new-rr-snp </div><div>1f2a5daccb16a783cfdcbfb77323948c test2-rr-snp</div><div>1f2a5daccb16a783cfdcbfb77323948c test2new-rr-snp</div></div><div><br></div><div>So bsdiff created a patch file that is only 228 bytes, but that's enough to completely reconstruct test2-rr-snp if you have test1-rr-snp.</div><div><br></div><div>-Brendan</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Nov 6, 2017 at 10:36 AM, Samuele Aicardi <span dir="ltr"><<a href="mailto:aicardi@eurecom.fr" target="_blank">aicardi@eurecom.fr</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="auto"><div>I'm trying with panda2 and I'm starting the recordings from the same snapshot, so they shouldn't be too different. Instead they seem quite both in size and content<div><div class="h5"><br><div class="gmail_extra"><br><div class="gmail_quote">On Nov 6, 2017 16:27, Brendan Dolan-Gavitt <<a href="mailto:brendandg@nyu.edu" target="_blank">brendandg@nyu.edu</a>> wrote:<br type="attribution"><blockquote class="m_2746706684377509944quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">If there are too many differences, then it doesn't make sense to store them as a diff. If all of the recordings come from the same base snapshot, you should find that they are very similar. This was the case in panda1, at least – are you trying this with panda1 or panda2?</div><div><br><div class="m_2746706684377509944elided-text">On Mon, Nov 6, 2017 at 10:20 AM, <span dir="ltr"><<a href="mailto:aicardi@eurecom.fr" target="_blank">aicardi@eurecom.fr</a>></span> wrote:<br><blockquote style="margin:0 0 0 0.8ex;border-left:1px #ccc solid;padding-left:1ex">And also, when I try to execute bdiff.cpp (which I previously compiled) I am always in the case 'if (diffs > 256) return 2;' with a lot of differences between the two snapshots.<div><div><br>
<br>
Quoting Brendan Dolan-Gavitt <<a href="mailto:brendandg@nyu.edu" target="_blank">brendandg@nyu.edu</a>>:<br>
<br>
<blockquote style="margin:0 0 0 0.8ex;border-left:1px #ccc solid;padding-left:1ex">
I should add that it *is* possible to extract the snapshot from a QCOW if<br>
that's what you really want. Just start up QEMU with:<br>
<br>
-loadvm snap -S<br>
<br>
Then at the monitor, do:<br>
<br>
migrate "exec:cat > reference-rr-snp"<br>
<br>
Best,<br>
Brendan<br>
<br>
On Mon, Nov 6, 2017 at 9:18 AM, Brendan Dolan-Gavitt <<a href="mailto:brendandg@nyu.edu" target="_blank">brendandg@nyu.edu</a>><br>
wrote:<br>
<br>
<blockquote style="margin:0 0 0 0.8ex;border-left:1px #ccc solid;padding-left:1ex">
Since they're all very similar to one another, you can just pick any<br>
-rr-snp to use as the reference snapshot. The pack_opt.sh script will<br>
detect if there's no sufficiently-similar reference snapshot and copy the<br>
current snapshot into the references directory.<br>
<br>
-Brendan<br>
<br>
On Mon, Nov 6, 2017 at 9:16 AM, <<a href="mailto:aicardi@eurecom.fr" target="_blank">aicardi@eurecom.fr</a>> wrote:<br>
<br>
<blockquote style="margin:0 0 0 0.8ex;border-left:1px #ccc solid;padding-left:1ex">
Thank you very much, I will try them soon!<br>
Just another question: how can I create the "reference" snapshot?<br>
Normally I start recording from a qcow2 image snaphsot that I've<br>
previously created with qemu monitor's "savevm <snap_name>" command,<br>
do I need to extract the reference snapshot from the qcow2 image?<br>
<br>
Regards,<br>
Samuele<br>
<br>
Quoting Brendan Dolan-Gavitt <<a href="mailto:brendandg@nyu.edu" target="_blank">brendandg@nyu.edu</a>>:<br>
<br>
The basic idea is very simple. The -rr-snp files differ from the<br>
<blockquote style="margin:0 0 0 0.8ex;border-left:1px #ccc solid;padding-left:1ex">
"reference" snapshots by only a few bytes, so you can just make a diff. I<br>
wrote to small programs to diff and patch the snapshots, bdiff and<br>
bpatch.py:<br>
<br>
<a href="http://giantpanda.gtisc.gatech.edu/malrec/rr/tools/" target="_blank">http://giantpanda.gtisc.gatech<wbr>.edu/malrec/rr/tools/</a><br>
<br>
There is also a script there that will create the diff and pack up a<br>
recording automatically given a snapshot and a list of possible reference<br>
snapshots:<br>
<br>
<a href="http://giantpanda.gtisc.gatech.edu/malrec/rr/tools/pack_opt.sh" target="_blank">http://giantpanda.gtisc.gatech<wbr>.edu/malrec/rr/tools/pack_opt.<wbr>sh</a><br>
<br>
Hope this helps!<br>
<br>
Best,<br>
Brendan<br>
<br>
<br>
<br>
On Mon, Nov 6, 2017 at 5:12 AM, <<a href="mailto:aicardi@eurecom.fr" target="_blank">aicardi@eurecom.fr</a>> wrote:<br>
<br>
Hello Brendan,<br>
<blockquote style="margin:0 0 0 0.8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
I am writing a script to apply my panda plugin on a large number of<br>
recordings.<br>
To do so I need to take a lot of recordings starting from the same<br>
qemu snapshot.<br>
My problem is that I don't have enough space to save all the *-rr-snp<br>
files on disk. I saw on this article<br>
(<a href="https://irfanulhaq.info/2015/12/09/replay-panda-malware-recordings/" target="_blank">https://irfanulhaq.info/2015/<wbr>12/09/replay-panda-malware-rec<wbr>ordings/</a>)<br>
that it's possible to save just a "patch" file containing only the<br>
differences from the original snapshot and then generate the actual<br>
*-rr-snp file only when it's needed.<br>
<br>
How can I produce such "patch" file?<br>
<br>
Thank you in advance,<br>
<br>
Samuele<br>
<br>
------------------------------<wbr>------------------------------<br>
-------------------<br>
This message was sent using EURECOM Webmail: <a href="http://webmail.eurecom.fr" target="_blank">http://webmail.eurecom.fr</a><br>
<br>
<br>
<br>
</blockquote>
<br>
--<br>
Brendan Dolan-Gavitt<br>
Assistant Professor, Department of Computer Science and Engineering<br>
NYU Tandon School of Engineering<br>
<br>
<br>
</blockquote>
------------------------------<wbr>------------------------------<br>
-------------------<br>
This message was sent using EURECOM Webmail: <a href="http://webmail.eurecom.fr" target="_blank">http://webmail.eurecom.fr</a><br>
<br>
<br>
</blockquote>
<br>
<br>
--<br>
Brendan Dolan-Gavitt<br>
Assistant Professor, Department of Computer Science and Engineering<br>
NYU Tandon School of Engineering<br>
<br>
</blockquote>
<br>
<br>
<br>
--<br>
Brendan Dolan-Gavitt<br>
Assistant Professor, Department of Computer Science and Engineering<br>
NYU Tandon School of Engineering<br>
<br>
</blockquote>
<br>
<br>
<br>
------------------------------<wbr>------------------------------<wbr>-------------------<br>
This message was sent using EURECOM Webmail: <a href="http://webmail.eurecom.fr" target="_blank">http://webmail.eurecom.fr</a><br>
<br>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div data-smartmail="gmail_signature">Brendan Dolan-Gavitt<br>Assistant Professor, Department of Computer Science and Engineering<br>NYU Tandon School of Engineering</div>
</div>
</blockquote></div><br></div></div></div></div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature">Brendan Dolan-Gavitt<br>Assistant Professor, Department of Computer Science and Engineering<br>NYU Tandon School of Engineering</div>
</div>