Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add CPU and Disk stats support to Illumos #378

Closed
wants to merge 19 commits into from
Closed

Add CPU and Disk stats support to Illumos #378

wants to merge 19 commits into from

Conversation

sean-
Copy link
Contributor

@sean- sean- commented Jun 1, 2017

Add support for various Illumos stats (via kstat(1) atm).

CC @jen20


func Usage(path string) (*UsageStat, error) {
stat := syscall.Statfs_t{}
err := syscall.Statfs(path, &stat)
stat := unix.Statfs_t{}
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm sorry, but could you tell me why use unix instead of syscall?
If use unix, at least darwin platform this error occured.

# github.com/shirou/gopsutil/disk
./disk_unix.go:17: cannot use stat (type unix.Statfs_t) as type syscall.Statfs_t in argument to getFsType

getFsType is this function

func getFsType(stat syscall.Statfs_t) string {
        return common.IntToString(stat.Fstypename[:])
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Drat, let me look into that. I'm mid-way through cherrypicking some patches onto this branch that will add memory support. I don't know why I didn't catch this during testing, however.

IMNSHO the syscall package should not be used anywhere where an x/sys/unix package variant is available because the syscall package is frozen, whereas the x/sys/unix package will track the current API.

In this particular case, there wasn't support for syscall.Statfs on the target platform, but was under the unix. package. It has been lamented a few times by the Go authors that the syscall package should be removed. IMO syscall would have been "fine" (on some level) if it weren't frozen. But since unix is there and not frozen, syscall should be outright deprecated and eschewed from all code bases so that programs can converge on a single maintained package (read: unix > syscall).

For instance, Illumos grew support for unix. Statvfs which didn't exist previously and couldn't have existed under the syscall package.
golang/sys@1e99a4f

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for my laziness since #30, we should move to x/sys/unix. I agree. However, I think this migration is not a part of this PR. So could you remove from this PR? Or, if x/sys/unix is required to work on illumos, I will make a new PR to migrate to x/sys/unix (but it takes a little time).

@sean-
Copy link
Contributor Author

sean- commented Jun 3, 2017

This PR is now based on top of #380. Once #380 is merged then I'll repush this branch.

@sean-
Copy link
Contributor Author

sean- commented Jun 6, 2017

@shirou this should be G2G to review and merge. The JSON parsing uses easyjson to generate the JSON parser so values can be decoded directly into their respective uint64 values. It's also faster and takes less memory according to the easyjson README.

@shirou
Copy link
Owner

shirou commented Jun 6, 2017

Could you revert some commits about rename *_solaris.go to *_illumos.go ? Filename is used to choose target by golang compiler, and solaris is reserved for GOOS (see here).

And, I support using easyjson in order to generate JSON faster, but I think it may be not included this PR. Could you separate?

@sean-
Copy link
Contributor Author

sean- commented Jun 6, 2017

@shirou All of the _illumos.go files have the required // +build solaris tags in order for Go to build the package correctly. These kstat(1M) values are not present on Solaris and are only available on Illumos. For example:

illumos/illumos-gate@9468939

As for easyjson, the only thing that's required to be checked in is the easyjson generated files, which have been added:

The tool is available via the tools target and regeneration of the generated files can be recreated via the generate target. The generate target will only be usable on platforms where GOOS=="solaris". I moved the generated files behind the necessary build tags so that non-Illumos variants won't carry around the extra code necessary to parse and won't be required to install easyjson.

Downstream consumers of shirou/gopsutil who are building on Illumos will now be required to vendor the following three packages:

  • github.com/mailru/easyjson
  • github.com/mailru/easyjson/jlexer
  • github.com/mailru/easyjson/jwriter

However that exercise is left to the consumer since shirou/gopsutil is a library and not an application.

Does that work/make sense?

@shirou
Copy link
Owner

shirou commented Jun 7, 2017

Well, I am not familiar with Solaris, but kstat is only on illumos, only one of a Solaris distribution? Since gopsutil aim to become widely used for multi platform and multi distribution, I think it is hard to add distribution (not platform) dependent functionality.

@sean-
Copy link
Contributor Author

sean- commented Jun 7, 2017

@shirou kstat is available on multiple Solaris distributions so this should work for most of the solaris build targets. Some of the counters are platform specific. I view this PR in the same vein as *_bsd.go using sysctl(1) to generate stats: the expectation is that there will be a need to add a freebsd.go or openbsd.go to address their platform-specific needs.

At present the worst that will happen is some of the values will be 0, however, the best that will happen is values will be non-zero and correct. Which is to say, this PR is strictly an improvement over the current state.

As issues arise, I'm happy to continue to try and address.

@shirou
Copy link
Owner

shirou commented Jun 7, 2017

openbsd, freebsd, netbsd and also dragonfly are called BSD family but have different GOOS target. not distribution.

@sean-
Copy link
Contributor Author

sean- commented Jun 7, 2017

@shirou right, but there isn't a GOOS yet for Illumos vs Solaris vs SmartOS. When that happens, I'm happy to revisit.

@sean-
Copy link
Contributor Author

sean- commented Jun 7, 2017

@shirou Until golang/go#20603 is accepted or finished, could you please review and merge this PR? Once the PR is merged I will back out the solaris build tags and will leave it up to a qualified member of the Solaris community to add the necessary build tags once they verify the functionality works.

@shirou
Copy link
Owner

shirou commented Jun 8, 2017

Understood. I leave this PR open. If there is progress, could you write comment here? Thanks.

sean- and others added 19 commits July 28, 2017 10:01
Memoize the paths to psrinfo(1) and isainfo(1).
This requires an updated `golang.org/x/sys/unix` to pick up `unix.Statvfs()`
for Illumos.
This implementation uses kstat(1) for the time being.
This will go away in the future but for now centralize the functionality
so it can be hunted down and removed when the time is right.
Add `VirtualMemoryZone()` and return `VirtualMemory()` in terms of
the values found in `VirtualMemoryZone()` when the measurement is
run inside of a zone.
@Smithx10
Copy link

Smithx10 commented Apr 4, 2019

What is the status of this? I've recently started diving into Nomad and SmartOS and require this functionality.

@shirou
Copy link
Owner

shirou commented Apr 12, 2019

#20603 is still not merged, so no one is working I think.

@Smithx10
Copy link

Smithx10 commented Jul 3, 2019

I'll open up a new PR for this when the new illumos tag is added to go 1.13 at the end of July.

@Lomanic
Copy link
Collaborator

Lomanic commented Sep 3, 2019

Go 1.13 has landed with support for the illumos GOOS build tag. Waiting for the corresponding PR @Smithx10 😉

Though I would prefer to not add a new external dependency (easyjson) just for illumos.

@Smithx10
Copy link

Smithx10 commented Sep 3, 2019

@Lomanic,

The latest news is that @jperkin was working on a PR. I backed off and am letting him do his thing.

@jperkin if you are short on cycles, maybe one of us could pick up the slack and get a PR in. Do you by chance have a branch out there?

Thank You,
Bruce Smith

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants