HALCS Usage¶
After installing all of the dependencies and HALCS itself to the desired versions, see HALCS Installation for more information, we can check if all of the needed parts are up and running.
Things to check¶
Device detection¶
The first thing to check is if your device is found by the OS. In the most typical usage of HALCS this would be a PCIe device.
If using the FPGA PCIe driver you can check if your device was enumerated by checking the /dev/fpga-<PCIe slot number> or /dev/fpga/<PCIe slot number>. If the board was detected a file should be created at /dev.
Another way to check if the board was detected is by looking at the
system logs, specifically in /var/log/messages, /var/log/dmesg or
via the dmesg
command. A common output of the FPGA PCIe driver
is shown below:
[Fri Feb 28 11:24:14 2020] pciehp 0000:02:08.0:pcie24: pending interrupts 0x0009 from Slot Status
[Fri Feb 28 11:24:14 2020] pciehp 0000:02:08.0:pcie24: Button pressed on Slot(8)
[Fri Feb 28 11:24:14 2020] pciehp 0000:02:08.0:pcie24: Card present on Slot(8)
[Fri Feb 28 11:24:14 2020] pciehp 0000:02:08.0:pcie24: pciehp_get_power_status: SLOTCTRL 80 value read 1431
[Fri Feb 28 11:24:14 2020] pciehp 0000:02:08.0:pcie24: PCI slot #8 - powering on due to button press
[Fri Feb 28 11:24:14 2020] pciehp 0000:02:08.0:pcie24: pciehp_get_power_status: SLOTCTRL 80 value read 1431
[Fri Feb 28 11:24:15 2020] pciehp 0000:02:08.0:pcie24: pending interrupts 0x0010 from Slot Status
[Fri Feb 28 11:24:15 2020] pciehp 0000:02:08.0:pcie24: pciehp_power_on_slot: SLOTCTRL 80 write cmd 0
[Fri Feb 28 11:24:15 2020] pciehp 0000:02:08.0:pcie24: __pciehp_link_set: lnk_ctrl = 0
[Fri Feb 28 11:24:15 2020] pciehp 0000:02:08.0:pcie24: pending interrupts 0x0100 from Slot Status
[Fri Feb 28 11:24:15 2020] pciehp 0000:02:08.0:pcie24: pciehp_check_link_active: lnk_status = 2041
[Fri Feb 28 11:24:15 2020] pciehp 0000:02:08.0:pcie24: slot(8): Link Up event
[Fri Feb 28 11:24:15 2020] pciehp 0000:02:08.0:pcie24: Link Up event ignored on slot(8): already powering on
[Fri Feb 28 11:24:15 2020] pciehp 0000:02:08.0:pcie24: pciehp_check_link_active: lnk_status = 2041
[Fri Feb 28 11:24:15 2020] pciehp 0000:02:08.0:pcie24: pciehp_check_link_status: lnk_status = 2041
[Fri Feb 28 11:24:15 2020] pci 0000:0b:00.0: [10ee:7014] type 00 class 0x118000
[Fri Feb 28 11:24:15 2020] pci 0000:0b:00.0: reg 0x10: [mem 0x00000000-0x000007ff 64bit]
[Fri Feb 28 11:24:15 2020] pci 0000:0b:00.0: reg 0x18: [mem 0x00000000-0x000fffff 64bit pref]
[Fri Feb 28 11:24:15 2020] pci 0000:0b:00.0: reg 0x20: [mem 0x00000000-0x0007ffff 64bit]
[Fri Feb 28 11:24:15 2020] pci 0000:0b:00.0: Max Payload Size set to 256 (was 128, max 512)
[Fri Feb 28 11:24:15 2020] pci 0000:0b:00.0: BAR 2: assigned [mem 0x91000000-0x910fffff 64bit pref]
[Fri Feb 28 11:24:15 2020] pci 0000:0b:00.0: BAR 4: assigned [mem 0x93900000-0x9397ffff 64bit]
[Fri Feb 28 11:24:15 2020] pci 0000:0b:00.0: BAR 0: assigned [mem 0x93980000-0x939807ff 64bit]
[Fri Feb 28 11:24:15 2020] pcieport 0000:02:08.0: PCI bridge to [bus 0b]
[Fri Feb 28 11:24:15 2020] pcieport 0000:02:08.0: bridge window [io 0xa000-0xafff]
[Fri Feb 28 11:24:15 2020] pcieport 0000:02:08.0: bridge window [mem 0x93900000-0x93cfffff]
[Fri Feb 28 11:24:15 2020] pcieport 0000:02:08.0: bridge window [mem 0x91000000-0x911fffff 64bit pref]
[Fri Feb 28 11:24:15 2020] pciDriver 0000:0b:00.0: probe for device 000b:0000
[Fri Feb 28 11:24:15 2020] pciDriver - pcidriver_probe : Found uTCA AMC board at 0000:0b:00.0
[Fri Feb 28 11:24:15 2020] pciDriver 0000:0b:00.0: enabling device (0000 -> 0002)
[Fri Feb 28 11:24:15 2020] pciDriver 0000:0b:00.0: Creating device name fpga-8
[Fri Feb 28 11:24:15 2020] pciDriver - pcidriver_probe_irq : Registered Interrupt Handler at pin 1, line 255, IRQ 16
[Fri Feb 28 11:24:15 2020] pciDriver - pcidriver_probe : pcidriver_probe: user_refcount = 1
[Fri Feb 28 11:24:15 2020] pciDriver - pcidriver_open : pcidriver_open: user_refcount = 2
[Fri Feb 28 11:24:15 2020] pciDriver - pcidriver_mmap : Entering mmap
[Fri Feb 28 11:24:15 2020] pciDriver - pcidriver_mmap_pci : Entering mmap_pci
[Fri Feb 28 11:24:15 2020] pciDriver - pcidriver_mmap : Entering mmap
[Fri Feb 28 11:24:15 2020] pciDriver - pcidriver_mmap_pci : Entering mmap_pci
[Fri Feb 28 11:24:15 2020] pciDriver - pcidriver_mmap : Entering mmap
[Fri Feb 28 11:24:15 2020] pciDriver - pcidriver_mmap_pci : Entering mmap_pci
[Fri Feb 28 11:24:15 2020] pciDriver - pcidriver_mmap : Entering mmap
[Fri Feb 28 11:24:15 2020] pciDriver - pcidriver_mmap_kmem : Entering mmap_kmem
[Fri Feb 28 11:24:15 2020] pciDriver - pcidriver_mmap_kmem : Got kmem_entry with id: 0
[Fri Feb 28 11:24:15 2020] pciDriver - pcidriver_mmap_kmem : Mapping address 44ba00000 / PFN 0044ba00
Device simple readout¶
If the driver is working ok, you can read/write to/from device registers
by using a simple program called regAccess
. This is shipped with
FPGA PCIe driver and is located at tests/pcie/bin
. From then
HALCS source tree you can located it at foreign/pcie-driver/tests/pcie/bin
.
The usage is as follows:
1 2 3 4 5 6 7 8 9 10 | Simple Register Access to FPGA
Usage: ./bin/regAccess [options]
-h --help Display this usage information
-b --devicefile <Device File> Device File
-v --verbose Verbose output
-r --read Perform read access
-w --write Perform write access
-n --barno <BAR number = [0|2|4] Bar Number
-a --address <FPGA Address in hex> Address to read/write
-d --data <FPGA Data in hex> Data to write to FPGA
|
In that case, if we want to read a register from board located at slot 1, at address 0x0 from BAR 4, which is the Wishbone register space in most cases, you could issue the following:
1 2 3 4 5 6 | $ ./bin/regAccess \
--devicefile /dev/fpga-1
--read \
--barno 4 \
--address 0x0
0x0000AA55
|
To write a value to register address 0x0
you could issue:
1 2 3 4 5 6 7 | $ ./bin/regAccess \
--devicefile /dev/fpga-1
--write \
--barno 4 \
--address 0x0 \
--data 0xAA55AA55
0xAA55AA55
|
SDB readout¶
On gateware projects that support SDB you can read its contents by
using the sdb-read-lnls
program located at foreign/libsdbfs/tools
.
The usage is as follows:
1 2 3 4 | sdb-read-lnls: Use: "sdb-read-lnls [options] <image-file> [<file>]
-l long listing (like ls -l)
-v verbose
-e <num> entry point offset
|
To read the SDB information from a board located at slot 11, from starting address
0x0
, you could issue:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 | $ sdb-read-lnls -l -e 0x0 /dev/fpga-11
INFO : [20-03-31 11:46:59] [ll_io] Ops set
INFO : [20-03-31 11:46:59] [ll_io] Created instance of llio
INFO : [20-03-31 11:46:59] [ll_io_pcie] Opened PCIe device located at /dev/fpga-11
sdb-read-lnls: listing format is to be defined
0000000000000651:e6a542c9 @ 00000000-007fffff WB4-Crossbar-GSI
0000000000000651:eef0b198 @ 00310000-00310fff WB4-Bridge-GSI
0000000000000651:e6a542c9 @ 00310000-003107ff WB4-Crossbar-GSI
1000000000001215:1bafbf1e @ 00310000-003100ff LNLS_POS_CALC_REGS
1000000000001215:12897592 @ 00310100-003101ff LNLS_BPM_SWAP
0000000000000651:eef0b198 @ 00320000-0032ffff WB4-Bridge-GSI
0000000000000651:e6a542c9 @ 00320000-00327fff WB4-Crossbar-GSI
1000000000001215:68e3b1af @ 00320000-003200ff LNLS_FMC250M_REGS
1000000000001215:2403f569 @ 00321000-003210ff LNLS_ACOMMON_REGS
0000000000000651:eef0b198 @ 00322000-00322fff WB4-Bridge-GSI
0000000000000651:e6a542c9 @ 00322000-003223ff WB4-Crossbar-GSI
1000000000001215:88c67d9c @ 00322000-003220ff LNLS_ACLK_REGS
000000000000ce42:123c5443 @ 00322100-003221ff WB-I2C-Master
000000000000ce42:e503947e @ 00322200-003222ff WB-SPI.Control
000000000000ce42:123c5443 @ 00323000-003230ff WB-I2C-Master
000000000000ce42:e503947e @ 00324000-003240ff WB-SPI.Control
000000000000ce42:e503947e @ 00325000-003250ff WB-SPI.Control
1000000000001215:4519a0ad @ 00330000-00330fff LNLS_BPM_ACQ_CORE
0000000000000651:eef0b198 @ 00340000-00340fff WB4-Bridge-GSI
0000000000000651:e6a542c9 @ 00340000-003407ff WB4-Crossbar-GSI
1000000000001215:1bafbf1e @ 00340000-003400ff LNLS_POS_CALC_REGS
1000000000001215:12897592 @ 00340100-003401ff LNLS_BPM_SWAP
0000000000000651:eef0b198 @ 00350000-0035ffff WB4-Bridge-GSI
0000000000000651:e6a542c9 @ 00350000-00357fff WB4-Crossbar-GSI
1000000000001215:68e3b1af @ 00350000-003500ff LNLS_FMC250M_REGS
1000000000001215:2403f569 @ 00351000-003510ff LNLS_ACOMMON_REGS
0000000000000651:eef0b198 @ 00352000-00352fff WB4-Bridge-GSI
0000000000000651:e6a542c9 @ 00352000-003523ff WB4-Crossbar-GSI
1000000000001215:88c67d9c @ 00352000-003520ff LNLS_ACLK_REGS
000000000000ce42:123c5443 @ 00352100-003521ff WB-I2C-Master
000000000000ce42:e503947e @ 00352200-003522ff WB-SPI.Control
000000000000ce42:123c5443 @ 00353000-003530ff WB-I2C-Master
000000000000ce42:e503947e @ 00354000-003540ff WB-SPI.Control
000000000000ce42:e503947e @ 00355000-003550ff WB-SPI.Control
1000000000001215:4519a0ad @ 00360000-00360fff LNLS_BPM_ACQ_CORE
0000000000000651:eef0b198 @ 00370000-00370fff WB4-Bridge-GSI
0000000000000651:e6a542c9 @ 00370000-003707ff WB4-Crossbar-GSI
000000000000ce42:8a5719ae @ 00370000-003700ff CERN_SIMPLE_UART
0000000000000651:35aa6b95 @ 00370100-003701ff GSI_GPIO_32
0000000000000651:35aa6b95 @ 00370200-003702ff GSI_GPIO_32
000000000000ce42:fdafb9dd @ 00370300-0037030f CERN_TICS_COUNTER
1000000000001215:51954750 @ 00380000-003800ff LNLS_AFCDIAG
1000000000001215:bcbb78d2 @ 00390000-003903ff LNLS_TRIGGER_IFACE
1000000000001215:84b6a5ac @ 00400000-004003ff LNLS_TRIGGER_MUX
1000000000001215:84b6a5ac @ 00410000-004103ff LNLS_TRIGGER_MUX
1000000000001215:7f9e3377 @ 00420000-00420fff LNLS_ACQ_CORE_PM
1000000000001215:7f9e3377 @ 00430000-00430fff LNLS_ACQ_CORE_PM
1000000000001215:84b6a5ac @ 00440000-004403ff LNLS_TRIGGER_MUX
1000000000001215:84b6a5ac @ 00450000-004503ff LNLS_TRIGGER_MUX
repo-url: https://github.com/lnls-dig/bpm-gw.git
synthesis-name: bpm-gw-sr-siriu+
commit-id: 4bf1f18fedb7a9694e8d5a80a859bc4e
tool-name: VIVADO
tool-version: 0x00020183
build-date: 20191206
build-user: LRusso
synthesis-name: dsp-cores+
commit-id: 31d42c16f00e75e3fa3d52ef2951ba71
synthesis-name: general-cores+
commit-id: b056d1983e92c723a20c78eecebea2cb
synthesis-name: infra-cores+
commit-id: dd80bd238f200c16e77f4125e31bc160
INFO : [20-03-31 11:46:59] [ll_io_pcie] Closed PCIe device located at /dev/fpga-11
INFO : [20-03-31 11:46:59] [ll_io] Ops unset
|
Malamute application¶
If the board was successfully detected, the next thing to check is the state of the Malamute daemon configuration file and state.
The daemon state can be check with the following command:
$ sudo systemctl status malamute
And the output should be something as the following:
* malamute.service - malamute service
Loaded: loaded (/usr/lib/systemd/system/malamute.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2020-03-10 12:24:52 -03; 3s ago
Main PID: 18844 (malamute)
CGroup: /system.slice/malamute.service
└─18844 /usr/bin/malamute -f /usr/etc/malamute/malamute.cfg
Mar 10 12:24:53 ia-testrabpmco-iocsrv malamute[18844]: client 705 address='8fd66a4a-7acd-79b8-fb12-1c6d80f61121' - closed connection
Mar 10 12:24:53 ia-testrabpmco-iocsrv malamute[18844]: client 705 address='8fd66a4a-7acd-79b8-fb12-1c6d80f61121' - de-registering
Mar 10 12:24:53 ia-testrabpmco-iocsrv malamute[18844]: client 716 address='469b0b4b-e012-2366-0693-ccd4e4391b81' - registering
Mar 10 12:24:53 ia-testrabpmco-iocsrv malamute[18844]: client 713 address='a2fb9c93-6d13-eb32-1ce2-1f7c3076ee54' - closed connection
Mar 10 12:24:53 ia-testrabpmco-iocsrv malamute[18844]: client 713 address='a2fb9c93-6d13-eb32-1ce2-1f7c3076ee54' - de-registering
Mar 10 12:24:53 ia-testrabpmco-iocsrv malamute[18844]: client 715 address='367d4945-8da9-9c22-658d-497016a61ca2' - closed connection
Mar 10 12:24:53 ia-testrabpmco-iocsrv malamute[18844]: client 715 address='367d4945-8da9-9c22-658d-497016a61ca2' - de-registering
Mar 10 12:24:53 ia-testrabpmco-iocsrv malamute[18844]: client 717 address='HALCS6:DEVIO:INIT0' - registering
Mar 10 12:24:53 ia-testrabpmco-iocsrv malamute[18844]: client 716 address='469b0b4b-e012-2366-0693-ccd4e4391b81' - closed connection
Mar 10 12:24:53 ia-testrabpmco-iocsrv malamute[18844]: client 716 address='469b0b4b-e012-2366-0693-ccd4e4391b81' - de-registering
The daemon should be in a running state as shown above and highlighted at the Active: line. Also, the configuration file in use by the daemon is shown after the -f option shown above and highlighted at the /usr/bin/malamute line.
If the application is not in a running state you can start it with:
$ sudo systemctl status malamute
Be sure that Malamute is enabled to start automatically. You can check if it is with:
$ sudo systemctl is-enabled malamute
And it should output:
enabled
If the output is not enabled you can enable it with:
$ sudo systemctl enable malamute
Lastly, the configuration file contents used by Malamute (the pathname given after the -f option) can be shown with:
$ cat /usr/etc/malamute/malamute.cfg
And it should show something like:
# Malamute configuration
# Apply to the whole broker
server
timeout = 10000 # Client connection timeout, msec
background = 0 # Run as background process
workdir = . # Working directory for daemon
verbose = 0
auth
verbose = 0
plain =
# Apply to the Malamute service
mlm_server
security
mechanism = null
echo = binding Malamute service to 'ipc:///tmp/malamute'
bind
endpoint = ipc:///tmp/malamute
service
queue
size-limit = max # Maximum size of service queue (in bytes)
size-warn = max # Warn if a service queue reaches this size
mailbox
size-limit = max # Maximum mailbox size (integer or "max")
size-warn = max # Warn if a mailbox reaches this size
HALCS application¶
The HALCS application state itself can also be checked with systemctl
commands. In order to do that, we must figure it out the correct HALCS
instance number.
For PCIe devices, it is given by the formula:
HALCS\_instance\_number = 2 * <board\_slot\_number> - 1
In which board slot number is the slot in which the board is plugged in. It usually is in the range [1...number\_of\_slots].
For other device types it could anything and it is application-specific.
Once the instance number is known, you can check its state with:
$ sudo systemctl status halcs-be@<HALCS_instance_number>
For example, in a system with a PCIe board using HALCS in slot 6, you would run:
$ sudo systemctl status halcs-be@11
And the output should be something like:
* halcs-be@11.service - HALCS server instance 11
Loaded: loaded (/etc/systemd/system/halcs-be@.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2020-03-10 12:24:52 -03; 1h 58min ago
Process: 18846 ExecStartPre=/bin/mkdir -p ${HALCS_LOG_DIR} (code=exited, status=0/SUCCESS)
Main PID: 18852 (halcsd)
CGroup: /system.slice/system-halcs\x2dbe.slice/halcs-be@11.service
└─18852 /usr/local/bin/halcsd -f /usr/local/etc/halcs/halcs.cfg -n be -t pcie -i 11 -b ipc:///tmp/malamute -l /var/log/halcs
Mar 10 12:24:52 ia-testrabpmco-iocsrv halcsd[18852]: INFO : [20-03-10 12:24:52] [hutils:utils] CFG hints hash key: "board11/halcs0", ...
Mar 10 12:24:52 ia-testrabpmco-iocsrv halcsd[18852]: INFO : [20-03-10 12:24:52] [hutils:utils] CFG hints hash key: "board11/halcs1", ...
Mar 10 12:24:52 ia-testrabpmco-iocsrv halcsd[18852]: INFO : [20-03-10 12:24:52] [hutils:utils] CFG hints hash key: "board12/halcs0", ...
Mar 10 12:24:52 ia-testrabpmco-iocsrv halcsd[18852]: INFO : [20-03-10 12:24:52] [hutils:utils] CFG hints hash key: "board12/halcs1", ...
Mar 10 12:24:52 ia-testrabpmco-iocsrv halcsd[18852]: INFO : [20-03-10 12:24:52] [halcsd] Dev_id parameter was set, but Dev_entry was not.
Mar 10 12:24:52 ia-testrabpmco-iocsrv halcsd[18852]: Defaulting Dev_entry to /dev/fpga-<dev_id>
Mar 10 12:24:52 ia-testrabpmco-iocsrv halcsd[18852]: INFO : [20-03-10 12:24:52] [halcsd] Dev_entry parameter was set to /dev/fpga-6.
Mar 10 12:24:52 ia-testrabpmco-iocsrv halcsd[18852]: INFO : [20-03-10 12:24:52] [halcsd] Dev_id parameter was set to 6.
Mar 10 12:24:52 ia-testrabpmco-iocsrv halcsd[18852]: INFO : [20-03-10 12:24:52] [halcsd] Smio_id parameter was set to 0.
Mar 10 12:24:52 ia-testrabpmco-iocsrv halcsd[18852]: I: 20-03-10 12:24:52 My address is 'HALCS6:DEVIO:AFC_DIAG0'
The important lines here are the same ones as the Systemd Malamute Status, the daemon status and the command-line executed.
With the HALCS application checked and in a running state everything should be in place for running client applications and communicating with this HALCS instance. For more information refer to HALCS Tools.
Lastly, the configuration file contents used by HALCS (the pathname given after the -f option) can be shown with:
$ cat /usr/local/etc/halcs/halcs.cfg
And it should show something like:
# HALCS configuration file
# Device manager configurations
dev_mngr
broker
bind = tcp://127.0.0.1:9999
log
dir = /media/remote_logs
filename = dev_mngr.log
verbose = 1 # Ask for a trace
daemonize = no # Ask for daemonize process (options are: yes or no)
workdir = . # Working directory for daemon
spawn_broker = no # Ask to spawn broker (options are: yes or no)
# Device I/O configurations
dev_io
board1
halcs0
dbe
board_type = afcv3.1
afe
bind =
proto =
halcs1
dbe
board_type = afcv3.1
afe
bind =
proto =
...
The configuration file would have an entry for each board slot board<board_slot>, and two entries for each instance of that slot halcs<instance_number>.