[panda-users] Replay of legacy records

Tue Sep 5 15:03:46 EDT 2017

Off the top of my head, did you include:

assert(init_osi_api());

immediately after:
panda_require("osi");
?

For example:
https://github.com/panda-re/panda/blob/060e90693f2ceb30b9c461a5835701e5c463b87a/panda/plugins/asidstory/asidstory.cpp#L359
(The same in PANDA 1.0 and 2.0)

HTH,
Adam

On Tue, 5 Sep 2017 at 10:40 <aicardi at eurecom.fr> wrote:

> Could you please tell me how to execute those recordings with
> qemu-system-x86_64 in 32-bit mode? I've tried to load the 'osi' plugin
> on several recordings but every time I got a segmentation fault.
>
> The way I execute them is:
> /home/samaicardi/panda1/qemu/x86_64-softmmu/qemu-system-x86_64 -replay
> <replay_name> -panda syscalls2:profile=windows7_x86 -panda
> <my_plugin>:<my_plugin_params> -os windows-32-7 -m 1G
>
> and in my_plugin I call:
> panda_require("osi");
>
>
> Thank you in advance,
> -samaicardi
>
> Quoting Brendan Dolan-Gavitt <brendandg at nyu.edu>:
>
> > The vast majority of those recordings are from Windows 7 32-bit, so osi
> > will work on them. They were recorded on an emulated x86_64 machine
> running
> > in 32-bit mode.
> >
> > On Mon, Sep 4, 2017 at 5:10 AM, <aicardi at eurecom.fr> wrote:
> >
> >> Thank you for the information, it worked.
> >>
> >> Is it possible to use the 'osi' plugin on those recordings? I've seen
> the
> >> introspection implemented only for windows 32 bit.
> >>
> >> -samaicardi
> >>
> >>
> >> Quoting Brendan Dolan-Gavitt <brendandg at nyu.edu>:
> >>
> >> The malware recordings use 1GB of RAM, so you need to pass "-m 1G" on
> the
> >>> command line when replaying.
> >>>
> >>> Also you may want to instead use the panda1 repository found here:
> >>>
> >>> https://github.com/moyix/panda
> >>>
> >>> As I think I've done a couple bugfixes to the old branch since we
> migrated
> >>> the repository to the new version of QEMU.
> >>>
> >>> -Brendan
> >>>
> >>> On Thu, Aug 31, 2017 at 11:56 AM, <aicardi at eurecom.fr> wrote:
> >>>
> >>> Ok I got it, thanks for the explanation.
> >>>>
> >>>> I have another problem actually, I tried to replay several records
> (from
> >>>> http://panda.gtisc.gatech.edu/malrec/) with the qemu-system-x86_64
> >>>> compiled from the branch called 'panda1' that I found here:
> >>>> https://github.com/panda-re/panda/tree/panda1
> >>>> I always get the following error:
> >>>> $> ~/panda1/qemu/x86_64-softmmu/qemu-system-x86_64 -replay
> >>>> logs/rr/7d114620-3e3c-4193-96ce-4689fd9efde3
> >>>>
> >>>> (process:1475): GLib-WARNING **:  /build/glib2.0-prJhLS/glib2.0-
> >>>> 2.48.2/./glib/gmem.c:483:
> >>>> custom memory allocation vtable not supported
> >>>> loading snapshot
> >>>> Block expected 134217728, found 1073741824, total 1082589184, system
> >>>> total
> >>>> 143065088
> >>>> qemu: warning: error while loading state for instance 0x0 of device
> 'ram'
> >>>> qemu-system-x86_64: Error -22 while loading VM state
> >>>> ... done.
> >>>> opening nondet log for read :   logs/rr/7d114620-3e3c-4193-96
> >>>> ce-4689fd9efde3-rr-nondet.log
> >>>> Infinite loop detected during replay, aborting.
> >>>> {guest_instr_count=0 pc=0x0000fff0, secondary=0x00000000}
> >>>> 7d114620-3e3c-4193-96ce-4689fd9efde3:           0 (  0.00%) instrs.
> >>>> 1.00 sec.  0.03 GB ram.
> >>>> total_instr in replay: 15418486377 <(541)%20848-6377>
> >>>> ERROR: replay failed!
> >>>> Time taken was: 0 seconds.
> >>>> max_queue_len = 1
> >>>> 0 items on recycle list, 0 bytes total
> >>>> ERROR: replay failed!
> >>>> Aborted (core dumped)
> >>>>
> >>>> Do you possibly know why every record seems to generate an infinite
> loop?
> >>>>
> >>>> Thanks in advance,
> >>>> samaicardi
> >>>>
> >>>>
> >>>>
> >>>> Quoting Brendan Dolan-Gavitt <brendandg at nyu.edu>:
> >>>>
> >>>> Unfortunately the new version is unlikely to ever be able to replay
> old
> >>>>
> >>>>> recordings; too much in QEMU has changed, most notably the underlying
> >>>>> default machine model (and hence the set of devices included in the
> >>>>> snapshot). We also took the opportunity to change some of the
> >>>>> record/replay
> >>>>> log entry types to better match QEMU's new memory API.
> >>>>>
> >>>>> It is frustrating, since we have 91,000 malware recordings now and it
> >>>>> would
> >>>>> be cool to use them in panda2, but for now malware-related work has
> to
> >>>>> use
> >>>>> panda1. I will be switching malrec over to panda2 as soon as I have
> some
> >>>>> free time, though.
> >>>>>
> >>>>> -Brendan
> >>>>>
> >>>>> On Thu, Aug 31, 2017 at 4:50 AM, <aicardi at eurecom.fr> wrote:
> >>>>>
> >>>>> Hello everyone,
> >>>>>
> >>>>>>
> >>>>>> I am writing a plugin for the new version of panda
> >>>>>> (https://github.com/panda-re/panda) and I would like to test it
> with
> >>>>>> several malware records that can be found here:
> >>>>>> http://panda.gtisc.gatech.edu/malrec/
> >>>>>>
> >>>>>> I followed the guidelines explained here:
> >>>>>> https://irfanulhaq.info/2015/12/09/replay-panda-malware-recordings/
> >>>>>> but I'm having troubles in starting the replays.
> >>>>>>
> >>>>>> When I try to execute one of those records I get the following error
> >>>>>> message:
> >>>>>> $> ~/panda2/x86_64-softmmu/qemu-system-x86_64 -replay
> >>>>>> ~/replays/malrec/logs/rr/bb67fd7e-7baa-437d-9333-9999b15f5fde
> >>>>>> > loading snapshot
> >>>>>> > qemu-system-x86_64: Unsupported migration stream version
> >>>>>> > Failed to load vmstate
> >>>>>> > Failed to start replay
> >>>>>>
> >>>>>> If I understood it properly, the 'problem' of those records is that
> >>>>>> they have been recorded starting from one of the snapshots that can
> be
> >>>>>> found here: http://panda.gtisc.gatech.edu/malrec/rr/references/
> >>>>>>
> >>>>>> These snapshots were taken using the old version of panda
> >>>>>> (https://github.com/moyix/panda).
> >>>>>>
> >>>>>> By analyzing the code of the new panda
> (include/migration/migration.h)
> >>>>>> I saw that there's the following line:
> >>>>>> #define QEMU_VM_FILE_VERSION         0x00000003
> >>>>>> which is different from what was declared in the old panda
> >>>>>> (qemu/savevm.c):
> >>>>>> #define QEMU_VM_SECTION_FULL         0x04
> >>>>>>
> >>>>>> That difference is causing the error I am getting and I may infer
> >>>>>> there are other differences between the two versions (for what
> >>>>>> concerns the procedure of saving a snapshot).
> >>>>>>
> >>>>>> My question is, since the two versions of panda take snapshots in
> >>>>>> different ways (they write different metadata I guess), is there a
> way
> >>>>>> to replay records (from http://panda.gtisc.gatech.edu/malrec/) with
> >>>>>> the new version of panda?
> >>>>>>
> >>>>>> Or, is it possible to 'patch' the vm snapshots (from
> >>>>>> http://panda.gtisc.gatech.edu/malrec/rr/references/) to make them
> work
> >>>>>> with the new version of panda?
> >>>>>>
> >>>>>> Thank you in advance for any suggestions you may have!
> >>>>>> samaicardi
> >>>>>>
> >>>>>> ------------------------------------------------------------
> >>>>>> -------------------
> >>>>>> This message was sent using EURECOM Webmail:
> http://webmail.eurecom.fr
> >>>>>>
> >>>>>>
> >>>>>> _______________________________________________
> >>>>>> panda-users mailing list
> >>>>>> panda-users at mit.edu
> >>>>>> http://mailman.mit.edu/mailman/listinfo/panda-users
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>>>> --
> >>>>> Brendan Dolan-Gavitt
> >>>>> Assistant Professor, Department of Computer Science and Engineering
> >>>>> NYU Tandon School of Engineering
> >>>>>
> >>>>>
> >>>>>
> >>>>
> >>>> ------------------------------------------------------------
> >>>> -------------------
> >>>> This message was sent using EURECOM Webmail:
> http://webmail.eurecom.fr
> >>>>
> >>>>
> >>>>
> >>>
> >>> --
> >>> Brendan Dolan-Gavitt
> >>> Assistant Professor, Department of Computer Science and Engineering
> >>> NYU Tandon School of Engineering
> >>>
> >>>
> >>
> >>
> >> ------------------------------------------------------------
> >> -------------------
> >> This message was sent using EURECOM Webmail: http://webmail.eurecom.fr
> >>
> >>
> >
> >
> > --
> > Brendan Dolan-Gavitt
> > Assistant Professor, Department of Computer Science and Engineering
> > NYU Tandon School of Engineering
> >
>
>
>
>
> -------------------------------------------------------------------------------
> This message was sent using EURECOM Webmail: http://webmail.eurecom.fr
>
>
> _______________________________________________
> panda-users mailing list
> panda-users at mit.edu
> http://mailman.mit.edu/mailman/listinfo/panda-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/pipermail/panda-users/attachments/20170905/270e5582/attachment.html