Skip to content

Commit

Permalink
sriov: add support
Browse files Browse the repository at this point in the history
Add support to report SRIOV devices.
Much like GPU devices, support is added using a new top-level package.

SRIOV devices are either Physical Functions or Virtual functions.
The preferred representation for ghw is Physical Functions, whose
dependent devices will be Virtual Functions; however, for the sake of
practicality, the API also exposes soft references to Virtual Functions,
so consumers of the API can access them directly and not navigating
the parent devices.

This patch also adds support in `ghwc`, to report the sriov information,
and in the `snapshot` package, to make sure to capture all the files
in sysfs that ghw cares about.

Last but not least, lacking access to suitable non-linux systems,
support is provided only on linux OS, even though the API tries hard
not to be linux-specific.

Resolves: jaypipes#92
Signed-off-by: Francesco Romani <fromani@redhat.com>
  • Loading branch information
ffromani committed Jun 25, 2021
1 parent 011ff34 commit 4715e44
Show file tree
Hide file tree
Showing 12 changed files with 643 additions and 14 deletions.
61 changes: 61 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1092,6 +1092,67 @@ information
`ghw.TopologyNode` struct if you'd like to dig deeper into the NUMA/topology
subsystem

### SRIOV

*This API is PROVISIONAL! ghw will try hard to not make breaking changes to this API, but still users are advice this new API is not
declared stable yet. We expect to declare it stable with ghw version 1.0.0*

SRIOV (Single-Root Input/Output Virtualization) is a class of PCI devices that ghw models explicitly, like gpus.

```go
package main

import (
"fmt"

"github.com/jaypipes/ghw"
)

func main() {
sriov, err := ghw.SRIOV()
if err != nil {
fmt.Fprintf(os.Stderr, "Error getting SRIOV info: %v", err)
}

fmt.Printf("%v\n", sriov)

for _, dev := range sriov.PhysicalFunctions {
fmt.Printf(" %v\n", dev)
}
}
```

ghw discovers the SRIOV devices starting from the Physical Function (PF) and exposes them in the `PhysicalFunctions` slice.
Virtual Function (VF) are exposed as properties of the PF instance with exposes them.
However, in some cases users are interested in the VFs first, so it's clumsy to navigate the PFs to learn about VFs.
To make this easier, ghw also exposes a slice of VF instances. These instances are soft references to the very same VF objects
you can find from the PF objects.

```go
package main

import (
"fmt"

"github.com/jaypipes/ghw"
)

func main() {
sriov, err := ghw.SRIOV()
if err != nil {
fmt.Fprintf(os.Stderr, "Error getting SRIOV info: %v", err)
}

fmt.Printf("%v\n", sriov)

// you will see the very same VF data data you seen from the previous example
for _, dev := range sriov.VirtualFunctions {
fmt.Printf(" %v\n", dev)
}
}
```


### Chassis

The host's chassis information is accessible with the `ghw.Chassis()` function. This
Expand Down
9 changes: 9 additions & 0 deletions alias.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ import (
"github.com/jaypipes/ghw/pkg/pci"
pciaddress "github.com/jaypipes/ghw/pkg/pci/address"
"github.com/jaypipes/ghw/pkg/product"
"github.com/jaypipes/ghw/pkg/sriov"
"github.com/jaypipes/ghw/pkg/topology"
)

Expand Down Expand Up @@ -149,3 +150,11 @@ type GraphicsCard = gpu.GraphicsCard
var (
GPU = gpu.New
)

type SRIOVInfo = sriov.Info
type PhysicalFunction = sriov.PhysicalFunction
type VirtualFunction = sriov.VirtualFunction

var (
SRIOV = sriov.New
)
48 changes: 48 additions & 0 deletions cmd/ghwc/commands/sriov.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
//
// Use and distribution licensed under the Apache license version 2.
//
// See the COPYING file in the root project directory for full text.
//

package commands

import (
"fmt"

"github.com/jaypipes/ghw"
"github.com/pkg/errors"
"github.com/spf13/cobra"
)

// sriovCmd represents the listing command
var sriovCmd = &cobra.Command{
Use: "sriov",
Short: "Show SRIOV devices information for the host system",
RunE: showSRIOV,
}

// showSRIOV show SRIOV physical device information for the host system.
func showSRIOV(cmd *cobra.Command, args []string) error {
sriov, err := ghw.SRIOV()
if err != nil {
return errors.Wrap(err, "error getting SRIOV info")
}

switch outputFormat {
case outputFormatHuman:
fmt.Printf("%v\n", sriov)

for _, dev := range sriov.PhysicalFunctions {
fmt.Printf(" %v\n", dev)
}
case outputFormatJSON:
fmt.Printf("%s\n", sriov.JSONString(pretty))
case outputFormatYAML:
fmt.Printf("%s", sriov.YAMLString())
}
return nil
}

func init() {
rootCmd.AddCommand(sriovCmd)
}
41 changes: 27 additions & 14 deletions host.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ package ghw

import (
"fmt"
"strings"

"github.com/jaypipes/ghw/pkg/context"

Expand All @@ -22,6 +23,7 @@ import (
"github.com/jaypipes/ghw/pkg/net"
"github.com/jaypipes/ghw/pkg/pci"
"github.com/jaypipes/ghw/pkg/product"
"github.com/jaypipes/ghw/pkg/sriov"
"github.com/jaypipes/ghw/pkg/topology"
)

Expand All @@ -40,6 +42,7 @@ type HostInfo struct {
Baseboard *baseboard.Info `json:"baseboard"`
Product *product.Info `json:"product"`
PCI *pci.Info `json:"pci"`
SRIOV *sriov.Info `json:"sriov"`
}

// Host returns a pointer to a HostInfo struct that contains fields with
Expand Down Expand Up @@ -91,6 +94,10 @@ func Host(opts ...*WithOption) (*HostInfo, error) {
if err != nil {
return nil, err
}
sriovInfo, err := sriov.New(opts...)
if err != nil {
return nil, err
}
return &HostInfo{
ctx: ctx,
CPU: cpuInfo,
Expand All @@ -104,26 +111,32 @@ func Host(opts ...*WithOption) (*HostInfo, error) {
Baseboard: baseboardInfo,
Product: productInfo,
PCI: pciInfo,
SRIOV: sriovInfo,
}, nil
}

// String returns a newline-separated output of the HostInfo's component
// structs' String-ified output
func (info *HostInfo) String() string {
return fmt.Sprintf(
"%s\n%s\n%s\n%s\n%s\n%s\n%s\n%s\n%s\n%s\n%s\n",
info.Block.String(),
info.CPU.String(),
info.GPU.String(),
info.Memory.String(),
info.Network.String(),
info.Topology.String(),
info.Chassis.String(),
info.BIOS.String(),
info.Baseboard.String(),
info.Product.String(),
info.PCI.String(),
)
var b strings.Builder
for _, s := range []fmt.Stringer{
info.Block,
info.CPU,
info.GPU,
info.Memory,
info.Network,
info.Topology,
info.Chassis,
info.BIOS,
info.Baseboard,
info.Product,
info.PCI,
info.SRIOV,
} {
b.WriteString(s.String())
b.WriteString("\n")
}
return b.String()
}

// YAMLString returns a string with the host information formatted as YAML
Expand Down
10 changes: 10 additions & 0 deletions pkg/pci/address/address.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,16 @@ type Address struct {
Function string
}

func (addr *Address) Equal(a *Address) bool {
if addr == nil && a == nil {
return true
}
if addr != nil && a != nil {
return addr.Domain == a.Domain && addr.Bus == a.Bus && addr.Device == a.Device && addr.Function == a.Function
}
return false
}

// String() returns the canonical [D]BDF representation of this Address
func (addr *Address) String() string {
return addr.Domain + ":" + addr.Bus + ":" + addr.Device + "." + addr.Function
Expand Down
8 changes: 8 additions & 0 deletions pkg/pci/address/address_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -80,3 +80,11 @@ func TestPCIAddressFromString(t *testing.T) {
}
}
}

func TestPCIAddressEqual(t *testing.T) {
addr1 := pciaddr.FromString("0000:03:00.A")
addr2 := pciaddr.FromString("03:00.A")
if addr1.Equal(addr2) == false {
t.Fatalf("addr1 %v and addr2 %v should be equal", addr1, addr2)
}
}
17 changes: 17 additions & 0 deletions pkg/snapshot/clonetree_pci_linux.go
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,15 @@ func scanPCIDeviceRoot(root string) (fileSpecs []string, pciRoots []string) {
"revision",
"vendor",
}

perDevEntriesOpt := []string{
"driver",
"net/*",
"physfn",
"sriov_*",
"virtfn*",
}

entries, err := ioutil.ReadDir(root)
if err != nil {
return []string{}, []string{}
Expand All @@ -95,6 +104,14 @@ func scanPCIDeviceRoot(root string) (fileSpecs []string, pciRoots []string) {
fileSpecs = append(fileSpecs, filepath.Join(pciEntry, perNetEntry))
}

for _, perNetEntryOpt := range perDevEntriesOpt {
netEntryOptPath := filepath.Join(pciEntry, perNetEntryOpt)
if items, err := filepath.Glob(netEntryOptPath); err == nil && len(items) > 0 {
fileSpecs = append(fileSpecs, netEntryOptPath)
}

}

if isPCIBridge(entryPath) {
trace("adding new PCI root %q\n", entryName)
pciRoots = append(pciRoots, pciEntry)
Expand Down
120 changes: 120 additions & 0 deletions pkg/sriov/sriov.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
//
// Use and distribution licensed under the Apache license version 2.
//
// See the COPYING file in the root project directory for full text.
//

package sriov

import (
"fmt"

"github.com/jaypipes/ghw/pkg/context"
"github.com/jaypipes/ghw/pkg/marshal"
"github.com/jaypipes/ghw/pkg/option"
"github.com/jaypipes/ghw/pkg/pci"
pciaddr "github.com/jaypipes/ghw/pkg/pci/address"
)

type Device struct {
Interfaces []string `json:"interfaces"`
// the PCI address where the SRIOV instance can be found
Address *pciaddr.Address `json:"address"`
PCI *pci.Device `json:"pci"`
}

func (d Device) ToString(devType string) string {
deviceStr := d.Address.String()
nodeStr := ""
if d.PCI != nil {
deviceStr = d.PCI.String()
if d.PCI.Node != nil {
nodeStr = fmt.Sprintf(" [affined to NUMA node %d]", d.PCI.Node.ID)
}
}
return fmt.Sprintf("%s function %s@%s", devType, nodeStr, deviceStr)
}

type PhysicalFunction struct {
Device
MaxVFNum int `json:"max_vf_num,omitempty"`
VFs []VirtualFunction `json:"vfs,omitempty"`
}

type VirtualFunction struct {
Device
ID int `json:"id"`
// Address of the (parent) Physical Function this Virtual Function pertains to.
ParentAddress *pciaddr.Address `json:"parent_address,omitempty"`
}

func (pf *PhysicalFunction) String() string {
return fmt.Sprintf("%s with %d/%d virtual functions",
pf.Device.ToString("physical"),
len(pf.VFs),
pf.MaxVFNum,
)
}

func (vf *VirtualFunction) String() string {
return fmt.Sprintf("%s index %d from %s",
vf.Device.ToString("virtual"),
vf.ID,
vf.ParentAddress,
)
}

type Info struct {
ctx *context.Context
// All the Physical Functions found in the host system,
PhysicalFunctions []*PhysicalFunction `json:"physical_functions,omitempty"`
// All the Virtual Functions found in the host system,
// This is the very same data found navigating the `PhysicalFunctions`;
// These pointers point back to the corresponding structs in the `PhysicalFunctions`
// slice.
VirtualFunctions []*VirtualFunction `json:"virtual_functions,omitempty"`
}

// New returns a pointer to an Info struct that contains information about the
// SRIOV devices on the host system.
func New(opts ...*option.Option) (*Info, error) {
return NewWithContext(context.New(opts...))
}

// New returns a pointer to an Info struct that contains information about the
// SRIOV devices on the host system, reusing a given context.
// Use this function when you want to consume this package from another,
// ensuring the two see a coherent set of resources.
func NewWithContext(ctx *context.Context) (*Info, error) {
info := &Info{ctx: ctx}
if err := ctx.Do(info.load); err != nil {
return nil, err
}
return info, nil
}

func (i *Info) String() string {
return fmt.Sprintf(
"sriov (%d phsyical %d virtual devices)",
len(i.PhysicalFunctions),
len(i.VirtualFunctions),
)
}

// simple private struct used to encapsulate SRIOV information in a top-level
// "sriov" YAML/JSON map/object key
type sriovPrinter struct {
Info *Info `json:"sriov,omitempty"`
}

// YAMLString returns a string with the SRIOV information formatted as YAML
// under a top-level "sriov:" key
func (i *Info) YAMLString() string {
return marshal.SafeYAML(i.ctx, sriovPrinter{i})
}

// JSONString returns a string with the SRIOV information formatted as JSON
// under a top-level "sriov:" key
func (i *Info) JSONString(indent bool) string {
return marshal.SafeJSON(i.ctx, sriovPrinter{i}, indent)
}
Loading

0 comments on commit 4715e44

Please sign in to comment.