Bryan Fu

Pipe With Powershell

2022-01-16T00:00:00+00:00

With Linux what we can do:

run_command < input.txt > output.txt

With Powershell on Windows

Get-content input.txt | run_command.exe | out-file output.txt

Ssh Tunneling Explained

2021-12-30T00:00:00+00:00

Excellent post of https://goteleport.com/blog/ssh-tunneling-explained/

What is SSH tunneling

https://goteleport.com/blog/images/2021/ssh-tunnel/ssh-tunnel.png

For example, during shell access, the data transmitted are binary streams detailing dimensions of pseudo-terminal and ASCII characters to run commands on the remote shell. However, during SSH port forwarding, the data transmitted can be a binary stream of protocol tunneled over SSH (e.g. SQL over SSH). So SSH tunneling is just a way to transport arbitrary data with a dedicated data stream (tunnel) inside an existing SSH session. This can be achieved with either local port forwarding, remote port forwarding, dynamic port forwarding, or by creating a TUN/TAP tunnel.

Freebsd Command

2021-09-06T00:00:00+00:00

sysctl command to find out how much RAM is installed on a FreeBSD

$ sysctl hw.physmem
$ sysctl hw | egrep 'hw.(phys|user|real)'

Or ·$ grep memory /var/run/dmesg.boot·

Direct Root Non Root Login

2019-03-08T00:00:00+00:00

Answer : The procedure described here disallows direct root login, so when you connect using SSH you need to first login as a normal user, then su to obtain root access.

Disabling root login
1. Edit the /etc/ssh/sshd_config file with a text editor and find the following line:

PermitRootLogin yes

Change the yes to no and remove the ‘#’ at the beginning of the line so that it reads :

PermitRootLogin no

Restart the sshd service:

# service sshd restart

Disabling direct root login for non-root users

Sometimes allowing all users the ability to remotely log onto a system can be a security risk. There are many ways to limit who can remotely access a system. You can use PAM, IPwrapers, or IPtables to name a few. However, one of the easiest ways to limit who can access a system via SSH is to configure the SSH daemon. The directive, AllowUsers, can be configured in /etc/ssh/sshd_config. This directive can be followed by the list of user name patterns, separated by spaces. If specified, login is allowed only for those user names.

AllowUsers [username]

Where [username] is the username you want to allow.

For example, to allow ssh login to users john and teena and disable for it for rest of the users, modify the AllowUsers directive as :

AllowUsers john teena

Restart the sshd service for the changes to take effect :

# service sshd restart

Allow / disallow groups ssh login

To restrict groups, the option AllowGroups and DenyGroups are used in the file /etc/ssh/sshd_config. The said options will allow or disallow users whose primary group or supplementary group matches one of the group patterns.

Network Trouble Shooting Tips

2018-10-29T00:00:00+00:00

dump packets go in and out to a NIC with specific MAC

tcpdump -A -vvv ether host 52:54:BE:8b:2f:92
tcpdump -i tap0 ether host 52:54:be:6b:01:12

###

How To Decide Host Is Virtual Or Physical

2018-09-27T00:00:00+00:00

For VMWare stuff:

https://kb.vmware.com/s/article/1009458

Testing the CPUID hypervisor present bit

int cpuid_check() 
{         
    unsigned int eax, ebx, ecx, edx;         
    char hyper_vendor_id[13];          
    cpuid(0x1, &eax, &ebx, &ecx, &edx;;         
    
    if  (bit 31 of ecx is set) {                 
        cpuid(0x40000000, &eax, &ebx, &ecx, &edx;;                 
        memcpy(hyper_vendor_id + 0, &ebx, 4);                 
        memcpy(hyper_vendor_id + 4, &ecx, 4);                 
        memcpy(hyper_vendor_id + 8, &edx, 4);                 
        hyper_vendor_id[12] = '\0';                 
        if (!strcmp(hyper_vendor_id, "VMwareVMware"))                         
        return 1;               // Success - running under VMware         
      }
      
    return 0; 
}

Testing the virtual BIOS DMI information and the hypervisor port

int dmi_check(void) {         
    char string[10];         
    GET_BIOS_SERIAL(string);          
    if (!memcmp(string, "VMware-", 7) || !memcmp(string, "VMW", 3))                 
        return 1;                       // DMI contains VMware specific string.         
    else                 
        return 0; 
}

Hypervisor port

#define VMWARE_HYPERVISOR_MAGIC 0x564D5868 
#define VMWARE_HYPERVISOR_PORT  0x5658  
#define VMWARE_PORT_CMD_GETVERSION      10  
#define VMWARE_PORT(cmd, eax, ebx, ecx, edx)                            \         
        __asm__("inl (%%dx)" :                                          \                         
        "=a"(eax), "=c"(ecx), "=d"(edx), "=b"(ebx) :                    \                         
        "0"(VMWARE_HYPERVISOR_MAGIC),                                   \                         
        "1"(VMWARE_PORT_CMD_##cmd),                                     \                         
        "2"(VMWARE_HYPERVISOR_PORT), "3"(UINT_MAX) :                    \                         
        "memory");  
        
int hypervisor_port_check(void) {         
    uint32_t eax, ebx, ecx, edx;         
    VMWARE_PORT(GETVERSION, eax, ebx, ecx, edx);         
    if (ebx == VMWARE_HYPERVISOR_MAGIC)                 
        return 1;               // Success - running under VMware         
    else                 
        return 0;     
}

Complete solution

int Detect_VMware(void) {         

    if (cpuid_check())                 
        return 1;               // Success running under VMware.         
    else if (dmi_check() && hypervisor_port_check())                 
        return 1;         
        
    return 0; 
} 

Unpack Repack Initial Ramdisk

2018-09-19T00:00:00+00:00

Unpack initial ramdisk in a new folder

# mkdir test
# cd test
# zcat /boot/initrd-2.6.18-164.6.1.el5.img | cpio -idmv

After making all necessary changes, use commands to repack and compress initrd image

# find . | cpio -o -c | gzip -9 > /boot/test.img

For image compressed with xz format, these commands are to extract image:

# mkdir /tmp/initrd
# cd /tmp/initrd
# xz -dc < initrd.img | cpio --quiet -i --make-directories 

Then to repack:

# cd /tmp/initrd
# find . 2>/dev/null | cpio --quiet -c -o | xz -9 --format=lzma >"new_initrd.img"

Create Pxe Server

2018-08-07T00:00:00+00:00

Prepare software packages

  sudo apt update
  apt install isc-dhcp-server
  apt install apache2 tftpd-hpa inetutils-inetd

  wget http://releases.ubuntu.com/xenial/ubuntu-16.04.4-server-amd64.iso
  mount -o loop ubuntu-16.04.4-server-amd64.iso /mnt/cdrom/
  cp -fr /mnt/cdrom/install/netboot/* /var/lib/tftpboot/
  mkdir /var/www/html/xenial-install
  cp -fr /mnt/cdrom/* /var/www/html/xenial-install/

Configure environment

  vim /etc/dhcp/dhcpd.conf
  vim /etc/default/tftpd-hpa 
  vim /etc/inetd.conf
  chmod +w /var/lib/tftpboot/pxelinux.cfg/default
  vim /var/lib/tftpboot/pxelinux.cfg/default

dhcpd.conf ``` ddns-update-style none; default-lease-time 600; max-lease-time 7200; log-facility local7;

A slightly different configuration for an internal subnet.

subnet 10.207.80.0 netmask 255.255.252.0 { range 10.207.82.135 10.207.82.136; allow booting; allow bootp; next-server 10.207.80.159; filename “pxelinux.0”; } option option-128 code 128 = string; option option-129 code 129 = text;

host vR730 { hardware ethernet 52:54:BE:6b:01:12; fixed-address 10.207.82.135; }


* tftpd-hpa

/etc/default/tftpd-hpa

TFTP_USERNAME=”tftp” TFTP_DIRECTORY=”/var/lib/tftpboot” TFTP_ADDRESS=”:69” TFTP_OPTIONS=”–secure” RUN_DAEMON=”yes” OPTIONS=”-l -s /var/lib/tftpboot”


* inetd.conf

tftp dgram udp wait root /usr/sbin/in.tftpd/usr/sbin/in.tftpd -s /var/lib/tftpboot


* /var/lib/tftpboot/pxelinux.cfg/default

D-I config version 2.0

search path for the c32 support libraries (libcom32, libutil etc.)

path ubuntu-installer/amd64/boot-screens/ include ubuntu-installer/amd64/boot-screens/menu.cfg default ubuntu-installer/amd64/boot-screens/vesamenu.c32 prompt 0 timeout 0

[linux] label linux kernel ubuntu-installer/amd64/linux append ks=http://10.207.80.159/ks.cfg vga=normal initrd=ubuntu-installer/amd64/initrd.gz ramdisk_size=16432 root=/dev/rd/0 rw –

  
* Start DHCP/TFTP/WWW services

375 systemctl restart tftpd-hpa 376 systemctl status tftpd-hpa

381 systemctl restart isc-dhcp-server 382 systemctl status isc-dhcp-server


Alternative commands

461 /etc/init.d/isc-dhcp-server start 463 /etc/init.d/tftpd-hpa start


### Debugging command

dhcpd -t /etc/dhcp/dhcpd.conf tcpdump -i ens32 > tcpdump.log tail -F /var/log/syslog ```

Error of cannot bind to local IPv4 socket: Address already in use

It seems like the problem is that the tftp port (69) is already in use, when you start the tftp server. This might be due to a new program which was installed/updated recently.

Running the following command will help you figure out which process is using tftp port (69) on your machine:

netstat -lnp | grep 69

Mount Qcow2

2018-07-04T00:00:00+00:00

How to mount a qcow2 disk image

This is a quick guide to mounting a qcow2 disk images on your host server. This is useful to reset passwords, edit files, or recover something without the virtual machine running.

Step 1 - Enable NBD on the Host

apt install qemu-utils 
modprobe nbd max_part=8

Step 2 - Connect the QCOW2 as network block device

qemu-nbd --connect=/dev/nbd0 /var/lib/vz/images/100/vm-100-disk-1.qcow2

Step 3 - Find The Virtual Machine Partitions

fdisk /dev/nbd0 -l

Running partx, that should create the /dev/nbd0p* device nodes (assuming they are indeed missing from /dev).

partx -a /dev/nbd0

Step 4 - Mount the partition from the VM

mount /dev/nbd0p1 /mnt/somepoint/

Step 5 - After you done, unmount and disconnect

    umount /mnt/somepoint/
    qemu-nbd --disconnect /dev/nbd0
    rmmod nbd

mount: unknown filesystem type LVM2_member

root@svennd:~# mount /dev/sdd2 /mnt/disk`

mount: unknown filesystem type 'LVM2_member'

The fdisk -l already told me its a LVM :

root@svennd:~# fdisk -l /dev/sdd
Disk /dev/sdd: 233.8 GiB, 251000193024 bytes, 490234752 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x0009345d
Device     Boot  Start       End   Sectors   Size Id Type
/dev/sdd1  *        63    208844    208782   102M 83 Linux
/dev/sdd2       208845 488247479 488038635 232.7G 8e Linux LVM

(/dev/sdi1 is /boot partition, /dev/sdi2 is where the /home data resides)

Seems lvm2 tools also provide a way to check if its lvm or not, using lvmdiskscan (/dev/sdd2 here)

root@svennd:~# lvmdiskscan
  /dev/sdb1  [       1.82 TiB]
  /dev/sdc2  [     149.04 GiB]
  /dev/sdd1  [     101.94 MiB]
  /dev/sdd2  [     232.71 GiB] LVM physical volume
  0 disks
  4 partitions
  0 LVM physical volume whole disks
  1 LVM physical volume

Fine, now lets scan what lv’s are to be found using lvscan

root@svennd:~# lvscan
 inactive '/dev/VolGroup00/LogVol00' [230.75 GiB] inherit
 inactive '/dev/VolGroup00/LogVol01' [1.94 GiB] inherit

Since this is a old disk in an enclosure, its not activated on system boot. So we need to “activate” this lvm volume.

root@svennd:~# vgchange -ay
 2 logical volume(s) in volume group "VolGroup00" now active

and bam, ready to mount :

root@svennd:~# lvscan
  ACTIVE            '/dev/VolGroup00/LogVol00' [230.75 GiB] inherit
  ACTIVE            '/dev/VolGroup00/LogVol01' [1.94 GiB] inherit

now to mount :

mount /dev/VolGroup00/LogVol00 /mnt/disk

succeed!

Powerful Socat

2018-07-04T00:00:00+00:00

The socat command shuffles data between two locations. One way to think of socat is as the cat command which transfers data between two locations rather than from a file to standard output. I say that socat works on two locations rather than two files because you can grab data from a network socket, named pipe, or even setup a general virtual network interface as one end point.

Socat is a tool to manipulate sockets, one input and one output. But the idea of sockets is too restrictive. The documentation speaks about “data channels” which can be combinations of:

a file
a pipe
a device (ex: a serial line)
a socket (IPv4, IPv6, raw, TCP, UDP, SSL)
a FD (STDIN, STDOUT)
a program or script

For each data channel, parameters can be added (port, speed, permissions, owners, etc). For those who use Netcat, the default features remain the same.

Example 1: To exchange data via a TCP session across two hosts:

  hosta$ socat TCP4-LISTEN:31337 OPEN:inputfile,creat,append
  hostb$ cat datafile | socat - TCP4:hosta:31337

Example 2: To use a local serial line (to configure a network device or access a modem) without a terminal emulator

  $ socat READLINE,history:/tmp/serial.cmds \
  OPEN:/dev/ttyS0,ispeed=9600,ospeed=9600,crnl,raw,sane,echo=false

The “READLINE” data channel uses GNU readline to allow editing and reusing input lines like a classic shell.

Example #3: To grab some HTTP content without a browser

  $ cat <<EOF | socat - TCP4:blog.rootshell.be:80
  GET / HTTP/1.1
  Host: blog.rootshell.be

  EOF

Example #4: To use Socat to collect Syslog messages

# socat -u UDP4-LISTEN:5140,reuseaddr,fork OPEN:/tmp/syslog.msg,creat,append

Example #5: The “EXEC” channel allow us to specify an external program or script. Using the “fdin=” and “fdout=” parameters, it is easy to parse the information received from the input channel and to send back information.

$ socat TCP4:12.34.56.78:31337 EXEC:parse.sh,fdin=3,fdout=4

The following Bash script simulates a web server and can look for suspicious content. If none is found, the visitor is redirected to another site. Note that, for security reasons, “EXEC” does not allow a relative path for the executable. It must be present in your $PATH.

  #!/bin/bash
  #
  # Simple example of honeypot running on HTTP
  # Usage: socat TCP4-LISTEN:80,reuseaddr,fork EXEC:honeypot.sh,fdin=3,fdout=4
  # FD 3 = incoming traffic
  # FD 4 = traffic sent back to the client
  #

  # Define the patterns for bad traffic here
  BADTRAFFIC1="../../.."
  BADTRAFFIC2="foobar"

  # Process the received HTTP headers
  while read -u 3 BUFFER
  do
    [ "$BUFFER" = "^M" ] && break
    echo $BUFFER | egrep -q -o "($BADTRAFFIC1|$BADTRAFFIC2)"
    if [ "$?" = "0" ]; then
      echo "ALERT: Suspicous HTTP: $BUFFER" >>http.log
      cat <END0 >&4
      --html content--
      END0
      exit 0
    fi
  done
  cat <<END1 >&4
  --html content--
  END1

By using the file descriptors 3 and 4, we can easily read what’s sent by the client and send data into the TCP session.

socat: The General Bidirectional Pipe Handler

Because socat allows bidirectional data flow between the two locations you specify, it doesn’t really matter which order you specify them in. Locations have the general form of TYPE:options where TYPE can be CREATE, GOPEN or OPEN for normal filesystem files. There are also shortcuts for some locations like STDIO (or just -) which reads and writes to standard input and output respectively.

The SYSTEM type can be used to execute a program and connect to its standard input and output. For example, the command shown below will run the date command and transfer its output to standard output.

$ socat SYSTEM:date -
Thu Apr 23 12:57:00 EST 2009

Many network services handle control commands using plain text. For example, SMTP servers, HTTP servers. The below socat command will open a connection to a Web server and fetch a page to the console. Notice that the port is specified using the service name and a comma separates the address from the cnrl option which handles line termination transformations for us.

$ socat - TCP:localhost:www,crnl
GET /

...

If the network service is more interactive, you might like to use readline to track your command history, improve command editing, and allow you to search and recall your previous commands. Instead of connecting standard IO as the first location in the above command, using READLINE,history=$HOME/.http_history will cause socat to use readline to get your commands.

Many of the socat location TYPEs take more than one option. For example, GOPEN (generic open) lets you specify append if you would like to append too rather than overwrite the file. The below keeps a log file of the time each time you execute it. This is similar to the Web server example, a comma separated list of additional options for the location.

$ date | socat - GOPEN:/tmp/capture,append

While this example is quite superfluous in that you could just use the shell » redirection to append to the file, you could also include a network link into the mix with minimal effort using socat as shown below. The first command connects port 3334 on localhost to the file /tmp/capture. The seek-end moves the file to zero bytes from the end and the append makes sure that bytes are appended to the file rather than overwriting it. The client command, shown as the second command below, is very similar to the simpler example shown above except we now send standard IO to a socket address.

$ socat TCP4-LISTEN:3334,reuseaddr,fork gopen:/tmp/capture,seek-end=0,append
$ date | socat STDIO tcp:localhost:3334

One great use case for socat is making device files from one machine available on another one. I’ll use the example from the socat manual page shown below to demonstrate. The first location creates a PTY device on the local machine allowing raw communication with the other location. The other location is an ssh connection to a server machine, where the standard IO is connected to the serial device on the remote machine.

(socat PTY,link=$HOME/dev/vmodem0,raw,echo=0,waitslave \
 EXEC:"ssh   modem-server.example.com socat - /dev/ttyS0,nonblock,raw,echo=0")

While creating virtual modems is not as attractive as it might once have been, other devices can be moved around too. The below command makes /dev/urandom from a server available through a named pipe on the local machine.

socat \
  PIPE:/tmp/test/foo  \
  SYSTEM:"ssh myserver socat - /dev/urandom" 

Creating a Virtual Private Network over SSH in a Single Line Virtual networks are created using the TUN device of the Linux kernel. Note that if you send data to a TUN device there is no encryption happening so if those packets move over the real network you have a Virtual Public Network. While there are overviews of using socat with TUN and socat with SSL I think it is much simpler to just use SSH to protect the network link from eavesdropping. You probably already have SSH setup so its much simpler to use because no SSL certificates need to be generated and distributed. The trick with using ssh is how to bolt things together. You could setup port forwarding with ssh and use socat to connect those ports to a virtual TUN device. But that leaves forwarded ports between the two hosts which serve no legitimate purpose other than servicing the socat TUN devices.

It is clear that one end point will be a direct TUN location, and the other is leaning towards being an ssh into the remote host. The trick is making the ssh into the remote host use socat to connect its standard IO to a TUN device. So we use socat twice in the one command: once to connect a TUN to an ssh session on the local machine, and once to connect standard IO to a TUN device on the remote end.

The below command will setup the 192.168.32.2 address on localhost to communicate with 192.168.32.1 on the server host over a VPN. If you use the 192.168.32.1 address you should be able to connect to network services on the server as though it was on the LAN.

The first location just sets up a local TUN device with an address and brings the network interface up. The second location will ssh into the server machine and run socat there to connect the standard IO of the ssh session to a TUN device on the server. The -d -d options can be selectively removed to remove the debugging chatter from the local and remote socat processes but are very informative when experimenting.

# socat -d -d  \
    TUN:192.168.32.2/24,up \
    SYSTEM:"ssh root@server socat -d -d  - 'TUN:192.168.32.1/24,up'" 

You might need to be root to create TUN devices. If socat can not make them as the current user you will see a message like the below.

2009/04/23 14:41:09 socat[17930] E ioctl(3, TUNSETIFF, {""}: Operation not permitted

socat is a great tool to have in your collective command line toolbox. There are options to use socat with tcpwrappers, and a huge array of the parameters that can be set on sockets and other through other low level system calls can be tweaked through parameters to socat.

The ability to setup a makeshift VPN using ssh for data protection using a one line command could be just what you are after when you want to get at a few services without needing to research which ports you need to forward.