[panda-users] How to create patches from memory snapshot

[Brendan Dolan-Gavitt]

Hmm, I just had a look and they are indeed very different. I don't have
time to debug this right now, but my guess is that the record-from-snapshot
feature isn't quite working correctly since we ported it to PANDA 2.

A simpler solution may just be to use a smarter binary diff program, such
as bsdiff. On two panda2 snapshots created from the same initial QCOW
snapshot, I got this:

moyix at lorenzo:~/git/panda/build$ ls -ld test[12]-rr-snp
-rw-rw---- 1 moyix moyix 84462802 Nov  6 10:42 test1-rr-snp
-rw-rw---- 1 moyix moyix 84454612 Nov  6 10:42 test2-rr-snp
moyix at lorenzo:~/git/panda/build$ bsdiff test1-rr-snp test2-rr-snp
test2.patch
moyix at lorenzo:~/git/panda/build$ ls -l test2.patch
-rw-rw-r-- 1 moyix moyix 228 Nov  6 10:53 test2.patch
moyix at lorenzo:~/git/panda/build$ bspatch test1-rr-snp test2new-rr-snp
test2.patch
moyix at lorenzo:~/git/panda/build$ md5sum test2-rr-snp test2new-rr-snp
1f2a5daccb16a783cfdcbfb77323948c  test2-rr-snp
1f2a5daccb16a783cfdcbfb77323948c  test2new-rr-snp

So bsdiff created a patch file that is only 228 bytes, but that's enough to
completely reconstruct test2-rr-snp if you have test1-rr-snp.

-Brendan

On Mon, Nov 6, 2017 at 10:36 AM, Samuele Aicardi <aicardi at eurecom.fr> wrote:

> I'm trying with panda2 and I'm starting the recordings from the same
> snapshot, so they shouldn't be too different. Instead they seem quite both
> in size and content
>
>
> On Nov 6, 2017 16:27, Brendan Dolan-Gavitt <brendandg at nyu.edu> wrote:
>
> If there are too many differences, then it doesn't make sense to store
> them as a diff. If all of the recordings come from the same base snapshot,
> you should find that they are very similar. This was the case in panda1, at
> least – are you trying this with panda1 or panda2?
>
> On Mon, Nov 6, 2017 at 10:20 AM, <aicardi at eurecom.fr> wrote:
>
> And also, when I try to execute bdiff.cpp (which I previously compiled) I
> am always in the case 'if (diffs > 256) return 2;' with a lot of
> differences between the two snapshots.
>
>
> Quoting Brendan Dolan-Gavitt <brendandg at nyu.edu>:
>
> I should add that it *is* possible to extract the snapshot from a QCOW if
> that's what you really want. Just start up QEMU with:
>
> -loadvm snap -S
>
> Then at the monitor, do:
>
> migrate "exec:cat > reference-rr-snp"
>
> Best,
> Brendan
>
> On Mon, Nov 6, 2017 at 9:18 AM, Brendan Dolan-Gavitt <brendandg at nyu.edu>
> wrote:
>
> Since they're all very similar to one another, you can just pick any
> -rr-snp to use as the reference snapshot. The pack_opt.sh script will
> detect if there's no sufficiently-similar reference snapshot and copy the
> current snapshot into the references directory.
>
> -Brendan
>
> On Mon, Nov 6, 2017 at 9:16 AM, <aicardi at eurecom.fr> wrote:
>
> Thank you very much, I will try them soon!
> Just another question: how can I create the "reference" snapshot?
> Normally I start recording from a qcow2 image snaphsot that I've
> previously created with qemu monitor's "savevm <snap_name>" command,
> do I need to extract the reference snapshot from the qcow2 image?
>
> Regards,
> Samuele
>
> Quoting Brendan Dolan-Gavitt <brendandg at nyu.edu>:
>
> The basic idea is very simple. The -rr-snp files differ from the
>
> "reference" snapshots by only a few bytes, so you can just make a diff. I
> wrote to small programs to diff and patch the snapshots, bdiff and
> bpatch.py:
>
> http://giantpanda.gtisc.gatech.edu/malrec/rr/tools/
>
> There is also a script there that will create the diff and pack up a
> recording automatically given a snapshot and a list of possible reference
> snapshots:
>
> http://giantpanda.gtisc.gatech.edu/malrec/rr/tools/pack_opt.sh
>
> Hope this helps!
>
> Best,
> Brendan
>
>
>
> On Mon, Nov 6, 2017 at 5:12 AM, <aicardi at eurecom.fr> wrote:
>
> Hello Brendan,
>
>
> I am writing a script to apply my panda plugin on a large number of
> recordings.
> To do so I need to take a lot of recordings starting from the same
> qemu snapshot.
> My problem is that I don't have enough space to save all the *-rr-snp
> files on disk. I saw on this article
> (https://irfanulhaq.info/2015/12/09/replay-panda-malware-recordings/)
> that it's possible to save just a "patch" file containing only the
> differences from the original snapshot and then generate the actual
> *-rr-snp file only when it's needed.
>
> How can I produce such "patch" file?
>
> Thank you in advance,
>
> Samuele
>
> ------------------------------------------------------------
> -------------------
> This message was sent using EURECOM Webmail: http://webmail.eurecom.fr
>
>
>
>
> --
> Brendan Dolan-Gavitt
> Assistant Professor, Department of Computer Science and Engineering
> NYU Tandon School of Engineering
>
>
> ------------------------------------------------------------
> -------------------
> This message was sent using EURECOM Webmail: http://webmail.eurecom.fr
>
>
>
>
> --
> Brendan Dolan-Gavitt
> Assistant Professor, Department of Computer Science and Engineering
> NYU Tandon School of Engineering
>
>
>
>
> --
> Brendan Dolan-Gavitt
> Assistant Professor, Department of Computer Science and Engineering
> NYU Tandon School of Engineering
>
>
>
>
> ------------------------------------------------------------
> -------------------
> This message was sent using EURECOM Webmail: http://webmail.eurecom.fr
>
>
>
>
> --
> Brendan Dolan-Gavitt
> Assistant Professor, Department of Computer Science and Engineering
> NYU Tandon School of Engineering
>
>
>


-- 
Brendan Dolan-Gavitt
Assistant Professor, Department of Computer Science and Engineering
NYU Tandon School of Engineering
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/pipermail/panda-users/attachments/20171106/6d2e0163/attachment-0001.html