How to interview security engineer
· One min read
OpSec
· One min read
Two-Factor Authentication (2FA)
Enable Two-Factor Authentication on every service, and keep your device with you.
FileVault
Enable FileVault to encrypt disk
danger
Lock Your Desktop When You're Away.
Seperate browsers for different purposes
For example, I usually used two browsers(Chrome Stable and Chrome Canary). Chrome Stable for my personal use, I will install any Chrome plugin I want on this browser. Chrome Canary for company use, without any Chrome plugin installed.
run-multiple-versions-of-chrome-side-by.html
Email
Register SPF for your domain to detect forging sender addresses
Privacy
Whenever we are using the software, some of them will collect your data. For example, search history.
Periodically check your data not leaked
GCP resource hierarchy
· 2 min read
Overview
Scenario:
- A company size of
200people seperated into10teams. - Administer can have the company-wide policy to limit GCP account.
- Easy way to switch user to the other team.
- Single login page for user to login
- Each team wants its own access to its resources.
- Also, different environment for specific purpose.
- Sometimes, they need temporarily cooperation between teams.
- For auditing, bills should be seperated by each team.
Solution:
- Google Group to grouping user by team or functionality.
- Nesting group to organize members within the team.
- GCP folder to have hiearchy architecture.
- Set IAM role by Google Group email
- Set Organization Policy on GCP folder limit resource.
- Using seperated GCP folder for different enviroment, producion, staging...... .
Organization
graph LR
R[Organization root] --> A[Team A folder]
R[Organization root] --> B[Team B folder]
A[Company folder] --> A1[Team A production folder]
A[Company folder] --> A2[Team A staging folder]
B[Company folder] --> B1[Team B production folder]
B[Company folder] --> B2[Team B staging folder]
R[Organization root] --> P[playground folder]
Nested Google Group
graph LR
subgraph A [Team A Google Group]
A1[Team A production Google Group]
A2[Team A staging Google Group]
end
IAM relationship between Google Group and Google project
graph LR
A[Team A Google Group] -- Folder Admin/Owner -->AF[Team A folder]
A1[Team A production Google Group] -- Folder Admin/Owner -->A1F[Team A production folder]
A2[Team A production Google Group] -- Folder Admin/Owner -->A2F[Team A staging folder]
P[playground Google Group] -- Folder Admin/Owner -->PF[Team B playground folder]
Terraform
- Seed project (Not Recommanded)
Create service account per GCP project as Terraform repo's credential.
graph LR;
A[A GCP project] --> AT[A Terraform repo];
AT[A Terraform repo] --> A[A GCP project];
B[B GCP project] --> BT[B Terraform repo];
BT[B Terraform repo] --> B[B GCP project];
KVM GPU passthrough Ubuntu 20.04
· 4 min read
Environment
- Host: Ubuntu 20.04
- Guest: Ubuntu 20.04
- GPU: NVIDIA® GeForce® RTX 2080 Ti
Enable IOMMU
Configure GRUB
Edit /etc/default/grub
# Intel CPU
GRUB_CMDLINE_LINUX_DEFAULT="intel_iommu=on"
# AMD CPU
GRUB_CMDLINE_LINUX_DEFAULT="amd_iommu=on iommu=pt kvm_amd.npt=1 kvm_amd.avic=1"
Update GRUB
sudo update-grub
Reboot
sudo shutdown -r now
Verify IOMMU is enabled
dmesg | grep IOMMU
Output:
IOMMU enabled
Enable IOMMU group
Check IOMMU group is enabled
for a in /sys/kernel/iommu_groups/*; do find $a -type l; done | sort --version-sort
output:
/sys/kernel/iommu_groups/0/devices/0000:00:00.0
/sys/kernel/iommu_groups/1/devices/0000:00:04.0
/sys/kernel/iommu_groups/2/devices/0000:00:04.1
/sys/kernel/iommu_groups/3/devices/0000:00:04.2
/sys/kernel/iommu_groups/4/devices/0000:00:04.3
Edit BIOS setting is not enabled
If output is not expected, configure BIOS setting
VT-d
(Asus)
- Advanced => System Agent Configuration => Intel VT for Directed I/O (VT-d)
(Supermicro)
Advanced => CPU Configuration => Intel Virtualization Technology => [enable]
Advanced => Chipset Configuration => North Bridge => IIO Configuration => Intel VT fot Directed I/O (VT-d) => [enable]
https://superuser.com/questions/1112238/intel-iommu-on-but-no-iommu-groups
Isolation of the guest GPU
graph LR
subgraph C [guest]
C1[PCI device]
end
subgraph B [hypervisor]
B1[VFIO] --> C1[PCI device]
end
subgraph A [Host]
A1[PCI device] --> B1[VFIO]
end
Using vfio-pci to manage PCI device
Find out vendor ID and device ID
lspci -nn | grep -i NVIDIA
01:00.0 VGA compatible controller [0300]: NVIDIA Corporation TU102 [GeForce RTX 2080 Ti] [10de:1e04] (rev a1)
01:00.1 Audio device [0403]: NVIDIA Corporation TU102 High Definition Audio Controller [10de:10f7] (rev a1)
01:00.2 USB controller [0c03]: NVIDIA Corporation TU102 USB 3.1 Host Controller [10de:1ad6] (rev a1)
01:00.3 Serial bus controller [0c80]: NVIDIA Corporation TU102 USB Type-C UCSI Controller [10de:1ad7] (rev a1)
GeForce RTX 2080 Ti VGA compatible controller:
PCI ID:01:00.0
vendor ID: 10de
device ID: 1e04
Configure GRUB
/etc/default/grub
Apply all the related devices
GRUB_CMDLINE_LINUX_DEFAULT="intel_iommu=on vfio-pci.ids=10de:1e04,10de:10f7,10de:1ad6,10de:1ad7"
Update GRUB
sudo update-grub
Reboot
sudo reboot
Verify PCI device is managed by vfio-pci
lspci -nnv
Find the line Kernel driver in use
0b:00.0 VGA compatible controller [0300]: NVIDIA Corporation TU102 [GeForce RTX 2080 Ti] [10de:1e04] (rev a1) (prog-if 00 [VGA controller])
...
Kernel driver in use: vfio-pci
Test GPU passthrough on kvm instance
Fresh install
Run virt-install with --host-device [device_id] and --features kvm_hidden=on parameters
virt-install ... \
--host-device 01:00.0 \
--features kvm_hidden=on \
Modify existing instance
virsh edit [domain]
Add PCI mapping hostdev block
0000:01:00.0 within the host will be mapped to 0000:04:00.0 within guest
danger
bus number should less than virtio's
Increase virtio's bus number to spare small number for new added entry
<devices>
...
<hostdev mode='subsystem' type='pci' managed='yes'>
<source>
<address domain='0x0000' bus='0x01' slot='0x00'function='0x0'/>
</source>
<address type='pci' domain='0x0000' bus='0x04'slot='0x00' function='0x0'/>
</hostdev>
<memballoon model='virtio'>
<address type='pci' domain='0x0000' bus='0x05'slot='0x00' function='0x0'/>
</memballoon>
<rng model='virtio'>
<backend model='random'>/dev/urandom</backend>
<address type='pci' domain='0x0000' bus='0x06'slot='0x00' function='0x0'/>
</rng>
</devices>
kvm hidden within features block
<features>
...
<kvm>
<hidden state='on'/>
</kvm>
</features>
Check GPU is working in guest
lspci
04:00.0 VGA compatible controller: NVIDIA Corporation TU102[GeForce RTX 2080 Ti] (rev a1)
Install NVIDIA driver
sudo apt update
sudo apt install nvidia-driver-460
sudo reboot
nvidia-smi
Wed Mar 10 08:19:43 2021
+---------------------------------------------------------------------------+
| NVIDIA-SMI 460.39 Driver Version: 460.39 CUDVersion: 11.2 |
|-------------------------------+--------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage GPU-Util Compute M. |
| | | MIG M. |
|===============================+====================+======================|
| 0 GeForce RTX 208... Off | 00000000:04:00.0 Of| N/A |
| 15% 44C P0 1W / 250W | 0MiB / 11019Mi| 0% Default |
| | | N/A |
+-------------------------------+--------------------+----------------------+
+---------------------------------------------------------------------------+
Processes: |
| GPU GI CI PID Type Procesname GPU Memory |
| ID ID Usage |
===========================================================================|
| No running processefound |
+---------------------------------------------------------------------------+
Reference
AWS resource hierarchy
· 3 min read
Overview
Scenario:
- A company size of
200people seperated into10teams. - Administer can have the company-wide policy to limit AWS account.
- Easy way to switch user to the other team.
- Single login page for user to login
- Each team wants its own access to its resources.
- Also, different environment for specific purpose.
- Sometimes, they need temporarily cooperation between teams.
- For auditing, bills should be seperated by each team.
Solution:
- AWS Organization to have hiearchy architecture.
- Move AWS Accounts into different Organizational Unit.
- Attach Service Control Policies to OU to limit AWS accounts.
- Using seperated account for different enviroment, producion, staging...... .
Organization
graph LR
R[Organization root] --> C[Company OU]
C[Company OU] --> A1[Team A production AWS account]
C[Company OU] --> A2[Team A staging AWS account]
C[Company OU] --> B1[Team B production AWS account]
C[Company OU] --> B2[Team B staging AWS account]
R[Organization root] --> P[playground AWS account]
Company OU will whitelist AWS resource and region. Playground account has no limitation on AWS resource. Periodically clean resource of playground account.
info
AWS Tag is another solution to have more detail bill within AWS account.
Terraform
Create terraform IAM access key and attach AdministratorAccess policy as Terraform repo's credential.
graph LR;
A[A AWS account] --> AT[A Terraform repo];
AT[A Terraform repo] --> A[A AWS account];
B[B AWS account] --> BT[B Terraform repo];
BT[B Terraform repo] --> B[B AWS account];
SSO
caution
Recommended using SSO to manage AWS Accounts, otherwise you will have to manage your account from different place.
Google apps SSO
danger
You have to have GSuite super administrator access.
Follow the How to Set Up Federated Single Sign-On to AWS Using Google Apps instruction
Then, you will find out it's one-to-one mapping between Google account and IAM role.
We want mapping of Google group and IAM role.
Refer repo https://github.com/1Strategy/sso-to-aws-using-gsuite
By using GSuite Admin API
Iterate the Google group, for each Google account, map to corresponding IAM role
AWS SSO
https://aws.amazon.com/tw/single-sign-on/
graph LR;
A[A AWS account admin group] --> AA[Administer role of A account];
A[A AWS account readonly group] --> AR[readonly role of account];
B[B AWS account admin group] --> BA[Administer role of A account];
B[B AWS account readonly group] --> BR[readonly role of account];
Assign user to group to have access to account.