Jekyll2025-04-24T08:33:35-07:00https://leimingyu.github.io/feed.xmlLeiming Yupersonal descriptionLeiming[email protected]Markdown Cheatsheet2024-12-10T00:00:00-08:002024-12-10T00:00:00-08:00https://leimingyu.github.io/posts/2024/12/markdown-cheatsheetMarkdown Cheatsheet 📒

Table of Contents

  1. Code Block

Code Block

]]>Leiming[email protected]Microsoft MX Format Emulation Tool on Ubuntu 22.042024-12-05T00:00:00-08:002024-12-05T00:00:00-08:00https://leimingyu.github.io/posts/2024/12/post-9Microscaling (MX) data formats are designed to reduce computational and storage costs in deep learning applications by combining a per-block scaling factor with narrow floating-point and integer types for individual elements. This approach balances hardware efficiency, model accuracy, and user convenience. Empirical results from popular benchmarks demonstrate the practicality of MX data formats as a drop-in replacement for standard FP32 in AI inference and training, with minimal user friction. Notably, MX formats have enabled the training of generative language models with sub-8-bit weights, activations, and gradients, incurring only minor accuracy losses without requiring modifications to the training process.

In 2023, the Open Compute Project (OCP) released the Microscaling Formats (MX) Specification, representing an initial effort to standardize an open, interoperable family of data formats with a shared, fine-grained block scale. This standardization effort includes contributions from industry leaders such as AMD, Arm, Intel, Meta, Microsoft, NVIDIA, and Qualcomm, aiming to create a unified approach to narrow precision data formats in AI workloads. To facilitate the adoption of MX data formats, Microsoft has developed a PyTorch emulation library that provides drop-in replacements for standard PyTorch modules and functions.

In this blog, I will go throught the step-by-step process of installing the Microxcaling repo using docker images. (repo link: https://github.com/microsoft/microxcaling)

Below are the desktop configurations.

OS: Ubuntu 20.04
NVIDIA-SMI 550.54.14
Driver Version: 550.54.14
CUDA Version: 12.4

Steps

  1. Install docker 
    $sudo apt update
$sudo apt install -y docker.io
  2. Install the docker image 
    $sudo docker pull nvcr.io/nvidia/pytorch:24.06-py3
  3. Install Install NVIDIA Container Toolkit (if not already installed) 
    $distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \
 && curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add - \
 && curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list
$sudo apt update
$sudo apt install -y nvidia-container-toolkit
  4. Log in the docker image 
    $sudo docker run --gpus all -it --rm -v $(pwd):/workspace nvcr.io/nvidia/pytorch:24.06-py3
  5. Update pip 
    $python -m pip install --upgrade pip
  6. Clone the repo. Update the requirements.txt file with the following versions. 
    torch==2.4.1
torchvision==0.19.1
torchaudio==2.4.1
  7. Install required packages 
    $pip install -r requirements.txt
  8. Set the PYTHONPATH Manually. If no errors occur, the path is set correctly.
$export PYTHONPATH=/workspace/microxcaling:$PYTHONPATH
$python -c "import mx"
  1. Run an exmaple test 
    /workspace/microxcaling/examples# sh run_mxfp6.sh
]]>Leiming[email protected]Setup NVIDIA Chat with RTX on Windows 112024-03-18T00:00:00-07:002024-03-18T00:00:00-07:00https://leimingyu.github.io/posts/2024/03/post-8“Chat with RTX” is an application that allows customization of a GPT large language model (LLM) with your specific content like documents and notes. This enables you to generate contextually relevant answers from a bespoke chatbot. The entire setup operates locally on RTX-enabled Windows PCs or workstations, ensuring quick and secure access to information.

I have an RTX 4060Ti and Windows 11 desktop. Below shows the steps to set it up.

Requirement

  • GPU NVIDIA GeForce™ RTX 30 or 40 Series GPU or NVIDIA RTX™ Ampere or Ada Generation GPU with at least 8GB of VRAM
  • RAM 16GB or greater
  • OS Windows 11
  • Driver 535.11 or later
  • File Size 35 GB

link : https://www.nvidia.com/en-us/ai-on-rtx/chat-with-rtx-generative-ai/

Setup

  1. Install visual studio
  2. Download cuda toolkit
  3. Install driver in toolkit. Reboot.
  4. Choose customize installation, install toolkit only. Disable : 
    - GeForce Experience Software
- PhysX
- Driver (alreadly installed)

    Example of output: ``` Installed:

    • Nsight for Visual Studio 2022
    • Nsight Monitor Not Installed:
    • Nsight for Visual Studio 2019 Reason: VS2019 was not found
    • Nsight for Visual Studio 2017 Reason: VS2017 was not found
    • Integrated Graphics Frame Debugger and Profiler Reason: see https://developer.nvidia.com/nsight-vstools
    • Integrated CUDA Profilers Reason: see https://developer.nvidia.com/nsight-vstools ```
  5. Install CUDNN 
    Download cudnn for windows (cuda 12.x).
Copy files to the nvidia tool kit folder (include/bin/lib files).
Reboot
  6. Install ChatWithRTX 
    Download ChatWithRTX
Run setup, select clean installation (it takes around 30-60 mins ...)
To run it, search "nvidia chat with rtx", select open
]]>Leiming[email protected]GPU Related2022-11-16T00:00:00-08:002022-11-16T00:00:00-08:00https://leimingyu.github.io/posts/2099/1/GPUnotesGPU-related notes

  1. cuda notes
]]>Leiming[email protected]Remote jupyter notebook via ssh port forwarding2022-11-07T00:00:00-08:002022-11-07T00:00:00-08:00https://leimingyu.github.io/posts/2022/11/post-7Remote jupyter notebook can be useful when the remote computer has better computing power than the local one or the remote is a server that you can connect from different places.

Prepare

  • First, you need to have openssh server installed on the remote machine.
  • You will need to generate public key authentication on our local machine.
  • In the remote machine, copy the content of id_rsa.pub to ~/.ssh/authorized_keys (reference link)
  • Test ssh without typing the password 
    $ssh <username>@<remoteIP>
  • In the remote machine, install anaconda or miniconda
  • Create a deeplearning env (optional) using conda
  • Activate deeplearning env, and install jupyter notebook (reference)
  • Configure Jupyter to use Python kernel 
    ipython kernel install --user --name=deeplearning
  • In the remote machine, add PATH for jupyter notebook in ~/.bashrc file. I am using the path in miniconda. 
    export PATH="/home/leiming/miniconda3/envs/deeplearning/bin:$PATH"

Test

  • Now, from your local machine, you can ssh to remote.

  • Start jupyter notebook via port 8889 in the remote machine. (Do not close the current terminal!) 
    jupyter notebook --no-browser --port 8889
  • Start a new terminal in your local terminal 
    ssh -N -L localhost:8899:localhost:8889 <username>@<remoteIP>
  • Copy the localhost link (from the remote machine terminal) to your browser in your local machine 
    http://localhost:8889/?token=dd5ac17bc80e068ece002ea35ccf867547030cfa0328
  • Now, you should see your brower has a jupyter notebook with the default deeplearning conda env.
]]>Leiming[email protected]Books Read in 20222022-11-06T00:00:00-07:002022-11-06T00:00:00-07:00https://leimingyu.github.io/posts/2022/12/ReadingNotesReading Notes

Computer Graphics

  1. Computer Graphics from Scratch
  2. Raytracing in one weekend(github)

Personal Growth

  1. The First 90 Days
  2. Master the Art of Self Discipline
  3. The Richest Man in Babylon
  4. Poor Charlie’s Almanack
  5. Mini Habits
  6. Who Moved My Cheese, Out of Maze
  7. OKR
  8. Rich Dad Poor Dad
  9. BehaviorTraps习惯陷阱
  10. SpeedWorkshop零秒工作
  11. McKinseyNegotiation麦肯锡高效沟通课_掌握高难度谈判的13种技巧
  12. TheRoadLessTraveled少有人走的路
  13. high EQ chat高情商聊天术
]]>Leiming[email protected]Linux Related2022-10-23T00:00:00-07:002022-10-23T00:00:00-07:00https://leimingyu.github.io/posts/2099/1/LinuxNotesLinux-related notes

  1. vimrc
  2. vim cheat sheet
  3. bashrc
  4. linux command sheet
]]>Leiming[email protected]Install NVIDIA Driver on CentOS 92022-08-01T00:00:00-07:002022-08-01T00:00:00-07:00https://leimingyu.github.io/posts/2022/8/post-6Installing the NVIDIA driver could be non-trivial if some packages or dependencies were missing during the installation process.

In this installation guide, I recorded the details to install NVIDIA driver 515.57 on CentOS 9.

]]>Leiming[email protected]Install Miniconda and Tensorflow GPU2022-02-10T00:00:00-08:002022-02-10T00:00:00-08:00https://leimingyu.github.io/posts/2022/2/blog-post-5miniconda is a simplified version of anaconda where you can configure different environments for various development purposes. Using this software, Tensorflow with GPU support can be set up for deep learning research.

In this tutorial,2022-rehl8-miniconda-tfgpu.pdf, I go over the following steps to set miniconda and tensorflow-gpu (v2.7.0) on RHEL 8.

  • Uninstall existing anaconda
  • Install miniconda for multi-users
  • Install tensorflow-gpu with pip
]]>Leiming[email protected]Use Screen in Linux2020-11-23T00:00:00-08:002020-11-23T00:00:00-08:00https://leimingyu.github.io/posts/2020/11/blog-post-4screen command in Linux provides the ability to launch and use multiple shell sessions from a single ssh session.

When you ssh connection is broken, the previous screen session continues to run on the remote machine. You won’t loose your work!

Install screen

On Ubuntu

$sudo apt-get install screen

On CentOS/Fedora

$sudo yum install screen

Common Commands

# list active sessions
$screen -ls

# create a new session
$screen -S <name>

# detach current working session
on your keyboard, press ctrl + a, then press d 

# resume an active session (if # of sessions > 1)
$screen -r <name>

# resume an active session (if the session is shown attached)
$screen -d -r <name>

# check the session name when you are using a screen session 
$echo $STY

# resume an active session (if # of sessions == 1)
$screen -x

# terminate current session 
type exit in the terminal, or  press ctrl + a, then k, then y

# terminate session outside the active one
use $screen -ls to list sessions (in the form of [pid].[sessionname]) 
find out the pid number
run $sudo kill [pid], to kill the session

#wipe a dead session
$screen -wipe

References:

  • https://kapeli.com/cheat_sheets/screen.docset/Contents/Resources/Documents/index
]]>Leiming[email protected]