[panda-users] Replay of legacy records

Brendan Dolan-Gavitt brendandg at nyu.edu
Wed Sep 6 10:26:02 EDT 2017 

That same command line and replay work fine for me. Could you provide a
backtrace from gdb of the crash?

On Wed, Sep 6, 2017 at 4:37 AM, <aicardi at eurecom.fr> wrote:

> Thank you for the suggestions.
> Yes, I did include 'assert(init_osi_api());' right after
> 'panda_require("osi");'. The problem is that the execution segfaults at the
> instruction 'panda_require("osi");', it doesn't reach the following
> instructions.
>
> When I try to execute the following command:
> /home/samaicardi/panda1/qemu/x86_64-softmmu/qemu-system-x86_64 -replay
> 6bb0dca4-0cea-465c-be82-5a39c1fed8ef -panda osi -panda win7x86intro
> -panda asidstory:width=180 -os windows-32-7 -m 1G
>
> I get:
> loading snapshot
> Unknown savevm section or instance 'vmmouse' 0
> ... done.
> opening nondet log for read :   ./6bb0dca4-0cea-465c-be82-5a3
> 9c1fed8ef-rr-nondet.log
> max_instr = 75766907229
> Segmentation fault (core dumped)
>
>
> Quoting Brendan Dolan-Gavitt <brendandg at nyu.edu>:
>
> Yep, good advice :) Also, you may want to verify that an existing plugin
>> that uses osi, such as asidstory, works with the replay.
>>
>> On Tue, Sep 5, 2017 at 3:03 PM, Bridgey theGeek <bridgeythegeek at gmail.com
>> >
>> wrote:
>>
>> Off the top of my head, did you include:
>>>
>>> assert(init_osi_api());
>>>
>>> immediately after:
>>> panda_require("osi");
>>> ?
>>>
>>> For example: https://github.com/panda-re/panda/blob/
>>> 060e90693f2ceb30b9c461a5835701e5c463b87a/panda/plugins/
>>> asidstory/asidstory.cpp#L359
>>> (The same in PANDA 1.0 and 2.0)
>>>
>>> HTH,
>>> Adam
>>>
>>>
>>> On Tue, 5 Sep 2017 at 10:40 <aicardi at eurecom.fr> wrote:
>>>
>>> Could you please tell me how to execute those recordings with
>>>> qemu-system-x86_64 in 32-bit mode? I've tried to load the 'osi' plugin
>>>> on several recordings but every time I got a segmentation fault.
>>>>
>>>> The way I execute them is:
>>>> /home/samaicardi/panda1/qemu/x86_64-softmmu/qemu-system-x86_64 -replay
>>>> <replay_name> -panda syscalls2:profile=windows7_x86 -panda
>>>> <my_plugin>:<my_plugin_params> -os windows-32-7 -m 1G
>>>>
>>>> and in my_plugin I call:
>>>> panda_require("osi");
>>>>
>>>>
>>>> Thank you in advance,
>>>> -samaicardi
>>>>
>>>> Quoting Brendan Dolan-Gavitt <brendandg at nyu.edu>:
>>>>
>>>> > The vast majority of those recordings are from Windows 7 32-bit, so
>>>> osi
>>>> > will work on them. They were recorded on an emulated x86_64 machine
>>>> running
>>>> > in 32-bit mode.
>>>> >
>>>> > On Mon, Sep 4, 2017 at 5:10 AM, <aicardi at eurecom.fr> wrote:
>>>> >
>>>> >> Thank you for the information, it worked.
>>>> >>
>>>> >> Is it possible to use the 'osi' plugin on those recordings? I've seen
>>>> the
>>>> >> introspection implemented only for windows 32 bit.
>>>> >>
>>>> >> -samaicardi
>>>> >>
>>>> >>
>>>> >> Quoting Brendan Dolan-Gavitt <brendandg at nyu.edu>:
>>>> >>
>>>> >> The malware recordings use 1GB of RAM, so you need to pass "-m 1G" on
>>>> the
>>>> >>> command line when replaying.
>>>> >>>
>>>> >>> Also you may want to instead use the panda1 repository found here:
>>>> >>>
>>>> >>> https://github.com/moyix/panda
>>>> >>>
>>>> >>> As I think I've done a couple bugfixes to the old branch since we
>>>> migrated
>>>> >>> the repository to the new version of QEMU.
>>>> >>>
>>>> >>> -Brendan
>>>> >>>
>>>> >>> On Thu, Aug 31, 2017 at 11:56 AM, <aicardi at eurecom.fr> wrote:
>>>> >>>
>>>> >>> Ok I got it, thanks for the explanation.
>>>> >>>>
>>>> >>>> I have another problem actually, I tried to replay several records
>>>> (from
>>>> >>>> http://panda.gtisc.gatech.edu/malrec/) with the qemu-system-x86_64
>>>> >>>> compiled from the branch called 'panda1' that I found here:
>>>> >>>> https://github.com/panda-re/panda/tree/panda1
>>>> >>>> I always get the following error:
>>>> >>>> $> ~/panda1/qemu/x86_64-softmmu/qemu-system-x86_64 -replay
>>>> >>>> logs/rr/7d114620-3e3c-4193-96ce-4689fd9efde3
>>>> >>>>
>>>> >>>> (process:1475): GLib-WARNING **:  /build/glib2.0-prJhLS/glib2.0-
>>>> >>>> 2.48.2/./glib/gmem.c:483:
>>>> >>>> custom memory allocation vtable not supported
>>>> >>>> loading snapshot
>>>> >>>> Block expected 134217728, found 1073741824, total 1082589184,
>>>> system
>>>> >>>> total
>>>> >>>> 143065088
>>>> >>>> qemu: warning: error while loading state for instance 0x0 of device
>>>> 'ram'
>>>> >>>> qemu-system-x86_64: Error -22 while loading VM state
>>>> >>>> ... done.
>>>> >>>> opening nondet log for read :   logs/rr/7d114620-3e3c-4193-96
>>>> >>>> ce-4689fd9efde3-rr-nondet.log
>>>> >>>> Infinite loop detected during replay, aborting.
>>>> >>>> {guest_instr_count=0 pc=0x0000fff0, secondary=0x00000000}
>>>> >>>> 7d114620-3e3c-4193-96ce-4689fd9efde3:           0 (  0.00%)
>>>> instrs.
>>>> >>>> 1.00 sec.  0.03 GB ram.
>>>> >>>> total_instr in replay: 15418486377 <(541)%20848-6377>
>>>>
>>>> >>>> ERROR: replay failed!
>>>> >>>> Time taken was: 0 seconds.
>>>> >>>> max_queue_len = 1
>>>> >>>> 0 items on recycle list, 0 bytes total
>>>> >>>> ERROR: replay failed!
>>>> >>>> Aborted (core dumped)
>>>> >>>>
>>>> >>>> Do you possibly know why every record seems to generate an infinite
>>>> loop?
>>>> >>>>
>>>> >>>> Thanks in advance,
>>>> >>>> samaicardi
>>>> >>>>
>>>> >>>>
>>>> >>>>
>>>> >>>> Quoting Brendan Dolan-Gavitt <brendandg at nyu.edu>:
>>>> >>>>
>>>> >>>> Unfortunately the new version is unlikely to ever be able to replay
>>>> old
>>>> >>>>
>>>> >>>>> recordings; too much in QEMU has changed, most notably the
>>>> underlying
>>>> >>>>> default machine model (and hence the set of devices included in
>>>> the
>>>> >>>>> snapshot). We also took the opportunity to change some of the
>>>> >>>>> record/replay
>>>> >>>>> log entry types to better match QEMU's new memory API.
>>>> >>>>>
>>>> >>>>> It is frustrating, since we have 91,000 malware recordings now and
>>>> it
>>>> >>>>> would
>>>> >>>>> be cool to use them in panda2, but for now malware-related work
>>>> has
>>>> to
>>>> >>>>> use
>>>> >>>>> panda1. I will be switching malrec over to panda2 as soon as I
>>>> have
>>>> some
>>>> >>>>> free time, though.
>>>> >>>>>
>>>> >>>>> -Brendan
>>>> >>>>>
>>>> >>>>> On Thu, Aug 31, 2017 at 4:50 AM, <aicardi at eurecom.fr> wrote:
>>>> >>>>>
>>>> >>>>> Hello everyone,
>>>> >>>>>
>>>> >>>>>>
>>>> >>>>>> I am writing a plugin for the new version of panda
>>>> >>>>>> (https://github.com/panda-re/panda) and I would like to test it
>>>> with
>>>> >>>>>> several malware records that can be found here:
>>>> >>>>>> http://panda.gtisc.gatech.edu/malrec/
>>>> >>>>>>
>>>> >>>>>> I followed the guidelines explained here:
>>>> >>>>>> https://irfanulhaq.info/2015/12/09/replay-panda-malware-
>>>> recordings/
>>>> >>>>>> but I'm having troubles in starting the replays.
>>>> >>>>>>
>>>> >>>>>> When I try to execute one of those records I get the following
>>>> error
>>>> >>>>>> message:
>>>> >>>>>> $> ~/panda2/x86_64-softmmu/qemu-system-x86_64 -replay
>>>> >>>>>> ~/replays/malrec/logs/rr/bb67fd7e-7baa-437d-9333-9999b15f5fde
>>>> >>>>>> > loading snapshot
>>>> >>>>>> > qemu-system-x86_64: Unsupported migration stream version
>>>> >>>>>> > Failed to load vmstate
>>>> >>>>>> > Failed to start replay
>>>> >>>>>>
>>>> >>>>>> If I understood it properly, the 'problem' of those records is
>>>> that
>>>> >>>>>> they have been recorded starting from one of the snapshots that
>>>> can be
>>>> >>>>>> found here: http://panda.gtisc.gatech.edu/malrec/rr/references/
>>>> >>>>>>
>>>> >>>>>> These snapshots were taken using the old version of panda
>>>> >>>>>> (https://github.com/moyix/panda).
>>>> >>>>>>
>>>> >>>>>> By analyzing the code of the new panda
>>>> (include/migration/migration.h)
>>>> >>>>>> I saw that there's the following line:
>>>> >>>>>> #define QEMU_VM_FILE_VERSION         0x00000003
>>>> >>>>>> which is different from what was declared in the old panda
>>>> >>>>>> (qemu/savevm.c):
>>>> >>>>>> #define QEMU_VM_SECTION_FULL         0x04
>>>> >>>>>>
>>>> >>>>>> That difference is causing the error I am getting and I may infer
>>>> >>>>>> there are other differences between the two versions (for what
>>>> >>>>>> concerns the procedure of saving a snapshot).
>>>> >>>>>>
>>>> >>>>>> My question is, since the two versions of panda take snapshots in
>>>> >>>>>> different ways (they write different metadata I guess), is there
>>>> a
>>>> way
>>>> >>>>>> to replay records (from http://panda.gtisc.gatech.edu/malrec/)
>>>> with
>>>> >>>>>> the new version of panda?
>>>> >>>>>>
>>>> >>>>>> Or, is it possible to 'patch' the vm snapshots (from
>>>> >>>>>> http://panda.gtisc.gatech.edu/malrec/rr/references/) to make
>>>> them
>>>> work
>>>> >>>>>> with the new version of panda?
>>>> >>>>>>
>>>> >>>>>> Thank you in advance for any suggestions you may have!
>>>> >>>>>> samaicardi
>>>> >>>>>>
>>>> >>>>>> ------------------------------------------------------------
>>>> >>>>>> -------------------
>>>> >>>>>> This message was sent using EURECOM Webmail:
>>>> http://webmail.eurecom.fr
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>> _______________________________________________
>>>> >>>>>> panda-users mailing list
>>>> >>>>>> panda-users at mit.edu
>>>> >>>>>> http://mailman.mit.edu/mailman/listinfo/panda-users
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>
>>>> >>>>> --
>>>> >>>>> Brendan Dolan-Gavitt
>>>> >>>>> Assistant Professor, Department of Computer Science and
>>>> Engineering
>>>> >>>>> NYU Tandon School of Engineering
>>>> >>>>>
>>>> >>>>>
>>>> >>>>>
>>>> >>>>
>>>> >>>> ------------------------------------------------------------
>>>> >>>> -------------------
>>>> >>>> This message was sent using EURECOM Webmail:
>>>> http://webmail.eurecom.fr
>>>> >>>>
>>>> >>>>
>>>> >>>>
>>>> >>>
>>>> >>> --
>>>> >>> Brendan Dolan-Gavitt
>>>> >>> Assistant Professor, Department of Computer Science and Engineering
>>>> >>> NYU Tandon School of Engineering
>>>> >>>
>>>> >>>
>>>> >>
>>>> >>
>>>> >> ------------------------------------------------------------
>>>> >> -------------------
>>>> >> This message was sent using EURECOM Webmail:
>>>> http://webmail.eurecom.fr
>>>> >>
>>>> >>
>>>> >
>>>> >
>>>> > --
>>>> > Brendan Dolan-Gavitt
>>>> > Assistant Professor, Department of Computer Science and Engineering
>>>> > NYU Tandon School of Engineering
>>>> >
>>>>
>>>>
>>>>
>>>> ------------------------------------------------------------
>>>> -------------------
>>>> This message was sent using EURECOM Webmail: http://webmail.eurecom.fr
>>>>
>>>>
>>>> _______________________________________________
>>>> panda-users mailing list
>>>> panda-users at mit.edu
>>>> http://mailman.mit.edu/mailman/listinfo/panda-users
>>>>
>>>>
>>> _______________________________________________
>>> panda-users mailing list
>>> panda-users at mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/panda-users
>>>
>>>
>>>
>>
>> --
>> Brendan Dolan-Gavitt
>> Assistant Professor, Department of Computer Science and Engineering
>> NYU Tandon School of Engineering
>>
>>
>
>
> ------------------------------------------------------------
> -------------------
> This message was sent using EURECOM Webmail: http://webmail.eurecom.fr
>
>


-- 
Brendan Dolan-Gavitt
Assistant Professor, Department of Computer Science and Engineering
NYU Tandon School of Engineering
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/pipermail/panda-users/attachments/20170906/14170065/attachment-0001.html

More information about the panda-users mailing list