Skip to content

add Python bindings to Libvmi events API#1

Merged
Wenzel merged 22 commits intomasterfrom
python_events
Jun 6, 2018
Merged

add Python bindings to Libvmi events API#1
Wenzel merged 22 commits intomasterfrom
python_events

Conversation

@Wenzel
Copy link
Copy Markdown
Member

@Wenzel Wenzel commented Apr 6, 2018

This PR aims to add support for Python bindings to Libvmi's events API.

The status is a big WIP that i wrote in a few hours, but so far i managed to have SingleStep events working, with the Python callback !

Example:

from libvmi import Libvmi, INIT_DOMAINNAME, INIT_EVENTS
from libvmi.event import SingleStepEvent

from pprint import pprint


def callback(vmi, event):
    pprint(event.to_dict())
    # increment
    event.data += 1

def main(args):
    if len(args) != 2:
        print('./singlestep-event.py <vm_name>')
        return 1

    vm_name = args[1]

    # register SIGINT
    signal.signal(signal.SIGINT, signal_handler)

    with Libvmi(vm_name, INIT_DOMAINNAME | INIT_EVENTS) as vmi:
        num_vcpus = vmi.get_num_vcpus()
        counter = 0
        ss_event = SingleStepEvent(range(num_vcpus), callback, data=counter)
        vmi.register_event(ss_event)
        # listen
        while not interrupted:
            print("Waiting for events")
            vmi.listen(500)
        print("Stop listening")

Events

I used a class hirarchy to create the Events, it was the most obvious thing to do for the implementation:

class Event(object)
    pass


class SingleStepEvent(Event)
    pass

class MemEvent(Event)
    pass

Callback

This was the tricky part.
The problem with CFFI is that you can only call a callback that you have already defined and compiled in the C library:
https://cffi.readthedocs.io/en/latest/using.html#extern-python-new-style-callbacks

That's why i defined a generic callback

@ffi.def_extern()
def generic_event_callback(cffi_vmi, cffi_event):

in events_cdef.h

// our generic callback
extern "Python" event_response_t generic_event_callback(
    vmi_instance_t vmi,
    vmi_event_t *event);

And i used the event.data to pass some extra data containing a handle to the Python Event object, which also contains the user defined Python callback:

Wrapping up into the vmi_event.data

class Event(object):

    def __init__(self,...):
        self.generic_data = {
            'vmi': None,
            'event': self,
        }
       self.py_callback = callback

   def get_callback(self):
       return self.py_callback

   def to_cffi(sell):
        # convert our generic_data dict to a CFFI void* handle
        # and keep a reference to the handle in self.generic_handle
        self.generic_handle = ffi.new_handle(self.generic_data)
        # assign the handle to the event data
        self.cffi_event.data = self.generic_handle
        self.cffi_event.callback = lib.generic_event_callback

This way i can get the handle back from the generic callback, and call the real callback:

@ffi.def_extern()
def generic_event_callback(cffi_vmi, cffi_event):
    # get generic event data dict
    generic_data = ffi.from_handle(cffi_event.data)
    # get true callback
    event = generic_data['event']
    vmi = generic_data['vmi']
    callback = event.get_callback()
    # call callback with the right Python objects as args
    event_response = callback(vmi, event)
    if not event_response:
        return 0
    return event_response

Results

you can test the example at examples/singlestep-event.py:

Waiting for events
{'data': 1807,
 'slat_id': 0,
 'type': 'SINGLESTEP',
 'vcpu_id': 0,
 'version': 4,
 'x86_regs': {'rax': '0xa800013a8',
              'rip': '0xfffff80002e00efd',
              'rsp': '0xfffff80003be7c28'}}
Waiting for events
{'data': 1808,
 'slat_id': 0,
 'type': 'SINGLESTEP',
 'vcpu_id': 0,
 'version': 4,
 'x86_regs': {'rax': '0x19e',
              'rip': '0xfffff80002e00f04',
              'rsp': '0xfffff80003be7c28'}}
Waiting for events
{'data': 1809,
 'slat_id': 0,
 'type': 'SINGLESTEP',
 'vcpu_id': 0,
 'version': 4,
 'x86_regs': {'rax': '0x19e',
              'rip': '0xfffff80002e00f07',
              'rsp': '0xfffff80003be7c28'}}
Waiting for events
{'data': 1810,
 'slat_id': 0,
 'type': 'SINGLESTEP',
 'vcpu_id': 0,
 'version': 4,
 'x86_regs': {'rax': '0x19e',
              'rip': '0xfffff80002e00f09',
              'rsp': '0xfffff80003be7c28'}}
Waiting for events
{'data': 1811,
 ^C'type': 'SINGLESTEP',
 'vcpu_id': 0,
 'version': 4,
 'x86_regs': {'rax': '0x19e',
              'rip': '0xfffff80002e00f0e',
              'rsp': '0xfffff80003be7c28'}}
Stop listening

Thanks.

@Wenzel Wenzel force-pushed the python_events branch 4 times, most recently from 89d5228 to 3da3fb0 Compare April 7, 2018 22:05
@Wenzel
Copy link
Copy Markdown
Member Author

Wenzel commented Apr 7, 2018

Update, i added support for memory events:

$ sudo ./venv/bin/python examples/memaccess-event.py xenwin7 NtOpenFile
... 
Waiting for events
{'data': {'symbol': 'NtOpenFile', 'symbol_addr': 18446735277662593612},
 'gla': '0xfffff80002bee24c',
 'gla_valid': True,
 'gptw': False,
 'in_access': 'X',
 'offset': '0x24c',
 'out_access': 'X',
 'slat_id': 0,
 'type': 'MEMORY',
 'vcpu_id': 0,
 'version': 4,
 'x86_regs': {'rax': '0x5',
              'rip': '0xfffff80002bee24c',
              'rsp': '0xfffff8800445bbe8'}}
Waiting for events
{'data': {'symbol': 'NtOpenFile', 'symbol_addr': 18446735277662593612},
 'gla': '0xfffff80002bee24c',
 'gla_valid': True,
 'gptw': False,
 'in_access': 'X',
 'offset': '0x24c',
 'out_access': 'X',
 'slat_id': 0,
 'type': 'MEMORY',
 'vcpu_id': 0,
 'version': 4,
 'x86_regs': {'rax': '0x5',
              'rip': '0xfffff80002bee24c',
              'rsp': '0xfffff8800445bbe8'}}
...

@Wenzel
Copy link
Copy Markdown
Member Author

Wenzel commented May 2, 2018

Update, add support for register events:

Waiting for events
{'data': None,
 'in_access': 'W',
 'out_access': 'INVALID',
 'previous': '0x187000',
 'slat_id': 0,
 'type': 'REGISTER',
 'value': '0x20e99000',
 'vcpu_id': 0,
 'version': 4,
 'x86_regs': {'rax': '0x0',
              'rip': '0xfffff8000288ae57',
              'rsp': '0xfffff880037d54e0'}}
Waiting for events
{'data': None,
 'in_access': 'W',
 'out_access': 'INVALID',
 'previous': '0x20e99000',
 'slat_id': 0,
 'type': 'REGISTER',
 'value': '0x18490000',
 'vcpu_id': 0,
 'version': 4,
 'x86_regs': {'rax': '0x0',
              'rip': '0xfffff8000288ae57',
              'rsp': '0xfffff88002862730'}}
Waiting for events
{'data': None,
 'in_access': 'W',
 'out_access': 'INVALID',
 'previous': '0x18490000',
 'slat_id': 0,
 'type': 'REGISTER',
 'value': '0x187000',
 'vcpu_id': 0,
 'version': 4,
 'x86_regs': {'rax': '0x0',
              'rip': '0xfffff8000288ae57',
              'rsp': '0xfffff80000b9cc00'}}
Waiting for events
^CStop listening

I have to investigate why the out_access is always INVALID 🤔

@tklengyel
Copy link
Copy Markdown
Contributor

Entirely possible we never actually set out_access for registers in the event processing code.

@Wenzel
Copy link
Copy Markdown
Member Author

Wenzel commented May 6, 2018

TODO:

  • add vmi_clear_event()
  • add support to return multiple (ORed) EventResponse values
  • implement generic memory event
  • review SingleStep constructor parameters

@Wenzel
Copy link
Copy Markdown
Member Author

Wenzel commented May 17, 2018

Rebased on master.

@Wenzel Wenzel force-pushed the python_events branch 2 times, most recently from d3a2ef4 to 33026f4 Compare May 22, 2018 10:43
@Wenzel
Copy link
Copy Markdown
Member Author

Wenzel commented May 22, 2018

I think this PR will just grow bigger and bigger if i want to implement all the LibVMI events interface.
Let's define small goals and improve it iteratively instead.
Plus, i would like to merge a PyTest PR beforehand, so i can implement some testing at the same time i'm adding new features.

postponed in a future PR:

  • Supporting multiple EventResponse
  • Generic memory events

@Wenzel
Copy link
Copy Markdown
Member Author

Wenzel commented May 22, 2018

This one is ready for review.
ping @smaresca , do you think you can take a look soon ?
what about @tklengyel , interested by reviewing some Python ? :)

@Wenzel
Copy link
Copy Markdown
Member Author

Wenzel commented May 28, 2018

@smaresca when do you think you can have time in the future to review this PR ?
I would like to merge it in order to write a few test in my test suite PR.

Also, don't mind the Travis CI build fail on /pr, it doesn't make sense, it's a Travis bug.

@Wenzel
Copy link
Copy Markdown
Member Author

Wenzel commented May 30, 2018

I added tests for single step and reg events.
However, for the memory access event, i don't want to rely on VMI_EVENT_RESPONSE_EMULATE, since the emulator is incomplete, and can crash the guest, even though the test should have passed.

I want instead to use the singlestepping instead of emulation.

The problem is that this technique is not 100% reliable yet.
I have situations where the guest is paused after receiving a single step event, for no reason:
see Wenzel/r2vmi#13 (comment).

We need to debug it.

@Wenzel
Copy link
Copy Markdown
Member Author

Wenzel commented Jun 6, 2018

Merging this one.

@Wenzel Wenzel merged commit a3749e8 into master Jun 6, 2018
@Wenzel Wenzel deleted the python_events branch June 6, 2018 09:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants