[panda-users] Replay of legacy records

Tue Sep 5 15:15:55 EDT 2017

Yep, good advice :) Also, you may want to verify that an existing plugin
that uses osi, such as asidstory, works with the replay.

On Tue, Sep 5, 2017 at 3:03 PM, Bridgey theGeek <bridgeythegeek at gmail.com>
wrote:

> Off the top of my head, did you include:
>
> assert(init_osi_api());
>
> immediately after:
> panda_require("osi");
> ?
>
> For example: https://github.com/panda-re/panda/blob/
> 060e90693f2ceb30b9c461a5835701e5c463b87a/panda/plugins/
> asidstory/asidstory.cpp#L359
> (The same in PANDA 1.0 and 2.0)
>
> HTH,
> Adam
>
>
> On Tue, 5 Sep 2017 at 10:40 <aicardi at eurecom.fr> wrote:
>
>> Could you please tell me how to execute those recordings with
>> qemu-system-x86_64 in 32-bit mode? I've tried to load the 'osi' plugin
>> on several recordings but every time I got a segmentation fault.
>>
>> The way I execute them is:
>> /home/samaicardi/panda1/qemu/x86_64-softmmu/qemu-system-x86_64 -replay
>> <replay_name> -panda syscalls2:profile=windows7_x86 -panda
>> <my_plugin>:<my_plugin_params> -os windows-32-7 -m 1G
>>
>> and in my_plugin I call:
>> panda_require("osi");
>>
>>
>> Thank you in advance,
>> -samaicardi
>>
>> Quoting Brendan Dolan-Gavitt <brendandg at nyu.edu>:
>>
>> > The vast majority of those recordings are from Windows 7 32-bit, so osi
>> > will work on them. They were recorded on an emulated x86_64 machine
>> running
>> > in 32-bit mode.
>> >
>> > On Mon, Sep 4, 2017 at 5:10 AM, <aicardi at eurecom.fr> wrote:
>> >
>> >> Thank you for the information, it worked.
>> >>
>> >> Is it possible to use the 'osi' plugin on those recordings? I've seen
>> the
>> >> introspection implemented only for windows 32 bit.
>> >>
>> >> -samaicardi
>> >>
>> >>
>> >> Quoting Brendan Dolan-Gavitt <brendandg at nyu.edu>:
>> >>
>> >> The malware recordings use 1GB of RAM, so you need to pass "-m 1G" on
>> the
>> >>> command line when replaying.
>> >>>
>> >>> Also you may want to instead use the panda1 repository found here:
>> >>>
>> >>> https://github.com/moyix/panda
>> >>>
>> >>> As I think I've done a couple bugfixes to the old branch since we
>> migrated
>> >>> the repository to the new version of QEMU.
>> >>>
>> >>> -Brendan
>> >>>
>> >>> On Thu, Aug 31, 2017 at 11:56 AM, <aicardi at eurecom.fr> wrote:
>> >>>
>> >>> Ok I got it, thanks for the explanation.
>> >>>>
>> >>>> I have another problem actually, I tried to replay several records
>> (from
>> >>>> http://panda.gtisc.gatech.edu/malrec/) with the qemu-system-x86_64
>> >>>> compiled from the branch called 'panda1' that I found here:
>> >>>> https://github.com/panda-re/panda/tree/panda1
>> >>>> I always get the following error:
>> >>>> $> ~/panda1/qemu/x86_64-softmmu/qemu-system-x86_64 -replay
>> >>>> logs/rr/7d114620-3e3c-4193-96ce-4689fd9efde3
>> >>>>
>> >>>> (process:1475): GLib-WARNING **:  /build/glib2.0-prJhLS/glib2.0-
>> >>>> 2.48.2/./glib/gmem.c:483:
>> >>>> custom memory allocation vtable not supported
>> >>>> loading snapshot
>> >>>> Block expected 134217728, found 1073741824, total 1082589184, system
>> >>>> total
>> >>>> 143065088
>> >>>> qemu: warning: error while loading state for instance 0x0 of device
>> 'ram'
>> >>>> qemu-system-x86_64: Error -22 while loading VM state
>> >>>> ... done.
>> >>>> opening nondet log for read :   logs/rr/7d114620-3e3c-4193-96
>> >>>> ce-4689fd9efde3-rr-nondet.log
>> >>>> Infinite loop detected during replay, aborting.
>> >>>> {guest_instr_count=0 pc=0x0000fff0, secondary=0x00000000}
>> >>>> 7d114620-3e3c-4193-96ce-4689fd9efde3:           0 (  0.00%) instrs.
>> >>>> 1.00 sec.  0.03 GB ram.
>> >>>> total_instr in replay: 15418486377 <(541)%20848-6377>
>> >>>> ERROR: replay failed!
>> >>>> Time taken was: 0 seconds.
>> >>>> max_queue_len = 1
>> >>>> 0 items on recycle list, 0 bytes total
>> >>>> ERROR: replay failed!
>> >>>> Aborted (core dumped)
>> >>>>
>> >>>> Do you possibly know why every record seems to generate an infinite
>> loop?
>> >>>>
>> >>>> Thanks in advance,
>> >>>> samaicardi
>> >>>>
>> >>>>
>> >>>>
>> >>>> Quoting Brendan Dolan-Gavitt <brendandg at nyu.edu>:
>> >>>>
>> >>>> Unfortunately the new version is unlikely to ever be able to replay
>> old
>> >>>>
>> >>>>> recordings; too much in QEMU has changed, most notably the
>> underlying
>> >>>>> default machine model (and hence the set of devices included in the
>> >>>>> snapshot). We also took the opportunity to change some of the
>> >>>>> record/replay
>> >>>>> log entry types to better match QEMU's new memory API.
>> >>>>>
>> >>>>> It is frustrating, since we have 91,000 malware recordings now and
>> it
>> >>>>> would
>> >>>>> be cool to use them in panda2, but for now malware-related work has
>> to
>> >>>>> use
>> >>>>> panda1. I will be switching malrec over to panda2 as soon as I have
>> some
>> >>>>> free time, though.
>> >>>>>
>> >>>>> -Brendan
>> >>>>>
>> >>>>> On Thu, Aug 31, 2017 at 4:50 AM, <aicardi at eurecom.fr> wrote:
>> >>>>>
>> >>>>> Hello everyone,
>> >>>>>
>> >>>>>>
>> >>>>>> I am writing a plugin for the new version of panda
>> >>>>>> (https://github.com/panda-re/panda) and I would like to test it
>> with
>> >>>>>> several malware records that can be found here:
>> >>>>>> http://panda.gtisc.gatech.edu/malrec/
>> >>>>>>
>> >>>>>> I followed the guidelines explained here:
>> >>>>>> https://irfanulhaq.info/2015/12/09/replay-panda-malware-
>> recordings/
>> >>>>>> but I'm having troubles in starting the replays.
>> >>>>>>
>> >>>>>> When I try to execute one of those records I get the following
>> error
>> >>>>>> message:
>> >>>>>> $> ~/panda2/x86_64-softmmu/qemu-system-x86_64 -replay
>> >>>>>> ~/replays/malrec/logs/rr/bb67fd7e-7baa-437d-9333-9999b15f5fde
>> >>>>>> > loading snapshot
>> >>>>>> > qemu-system-x86_64: Unsupported migration stream version
>> >>>>>> > Failed to load vmstate
>> >>>>>> > Failed to start replay
>> >>>>>>
>> >>>>>> If I understood it properly, the 'problem' of those records is that
>> >>>>>> they have been recorded starting from one of the snapshots that
>> can be
>> >>>>>> found here: http://panda.gtisc.gatech.edu/malrec/rr/references/
>> >>>>>>
>> >>>>>> These snapshots were taken using the old version of panda
>> >>>>>> (https://github.com/moyix/panda).
>> >>>>>>
>> >>>>>> By analyzing the code of the new panda
>> (include/migration/migration.h)
>> >>>>>> I saw that there's the following line:
>> >>>>>> #define QEMU_VM_FILE_VERSION         0x00000003
>> >>>>>> which is different from what was declared in the old panda
>> >>>>>> (qemu/savevm.c):
>> >>>>>> #define QEMU_VM_SECTION_FULL         0x04
>> >>>>>>
>> >>>>>> That difference is causing the error I am getting and I may infer
>> >>>>>> there are other differences between the two versions (for what
>> >>>>>> concerns the procedure of saving a snapshot).
>> >>>>>>
>> >>>>>> My question is, since the two versions of panda take snapshots in
>> >>>>>> different ways (they write different metadata I guess), is there a
>> way
>> >>>>>> to replay records (from http://panda.gtisc.gatech.edu/malrec/)
>> with
>> >>>>>> the new version of panda?
>> >>>>>>
>> >>>>>> Or, is it possible to 'patch' the vm snapshots (from
>> >>>>>> http://panda.gtisc.gatech.edu/malrec/rr/references/) to make them
>> work
>> >>>>>> with the new version of panda?
>> >>>>>>
>> >>>>>> Thank you in advance for any suggestions you may have!
>> >>>>>> samaicardi
>> >>>>>>
>> >>>>>> ------------------------------------------------------------
>> >>>>>> -------------------
>> >>>>>> This message was sent using EURECOM Webmail:
>> http://webmail.eurecom.fr
>> >>>>>>
>> >>>>>>
>> >>>>>> _______________________________________________
>> >>>>>> panda-users mailing list
>> >>>>>> panda-users at mit.edu
>> >>>>>> http://mailman.mit.edu/mailman/listinfo/panda-users
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>
>> >>>>> --
>> >>>>> Brendan Dolan-Gavitt
>> >>>>> Assistant Professor, Department of Computer Science and Engineering
>> >>>>> NYU Tandon School of Engineering
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>
>> >>>> ------------------------------------------------------------
>> >>>> -------------------
>> >>>> This message was sent using EURECOM Webmail:
>> http://webmail.eurecom.fr
>> >>>>
>> >>>>
>> >>>>
>> >>>
>> >>> --
>> >>> Brendan Dolan-Gavitt
>> >>> Assistant Professor, Department of Computer Science and Engineering
>> >>> NYU Tandon School of Engineering
>> >>>
>> >>>
>> >>
>> >>
>> >> ------------------------------------------------------------
>> >> -------------------
>> >> This message was sent using EURECOM Webmail: http://webmail.eurecom.fr
>> >>
>> >>
>> >
>> >
>> > --
>> > Brendan Dolan-Gavitt
>> > Assistant Professor, Department of Computer Science and Engineering
>> > NYU Tandon School of Engineering
>> >
>>
>>
>>
>> ------------------------------------------------------------
>> -------------------
>> This message was sent using EURECOM Webmail: http://webmail.eurecom.fr
>>
>>
>> _______________________________________________
>> panda-users mailing list
>> panda-users at mit.edu
>> http://mailman.mit.edu/mailman/listinfo/panda-users
>>
>
> _______________________________________________
> panda-users mailing list
> panda-users at mit.edu
> http://mailman.mit.edu/mailman/listinfo/panda-users
>
>

-- 
Brendan Dolan-Gavitt
Assistant Professor, Department of Computer Science and Engineering
NYU Tandon School of Engineering
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/pipermail/panda-users/attachments/20170905/29cab50b/attachment-0001.html