Skip to content

Commit

Permalink
[dev.regabi] all: merge master (bf0f7c9) into dev.regabi
Browse files Browse the repository at this point in the history 
This merge involved two merge conflicts:

1. walk's ascompatee code has been substantially refactored on
dev.regabi, so CL 285633 is ported to the new style.

2. The os.TestDirFS workaround added in CL 286213 can be removed now
that #42637 has been fixed by CL 285720.

Conflicts:

- src/cmd/compile/internal/gc/walk.go
- src/os/os_test.go

Merge List:

+ 2021-01-25 bf0f7c9 doc/go1.16: mention os.DirFS in os section
+ 2021-01-25 deaf29a cmd/compile: fix order-of-assignment issue w/ defers
+ 2021-01-25 ad2ca26 doc/go1.16: mention os.DirEntry and types moved from os to io/fs
+ 2021-01-25 a51921f doc/go1.16: mention new testing/iotest functions
+ 2021-01-25 e6b6d10 doc/go1.16: mention deprecation of io/ioutil
+ 2021-01-25 96a2763 doc/go1.16: mention go/build changes
+ 2021-01-25 3d85c69 html/template: revert "avoid race when escaping updates template"
+ 2021-01-25 54514c6 cmd/go: fix TestScript/cgo_path, cgo_path_space when CC set
+ 2021-01-25 6de8443 doc/asm: add a section on go_asm.h, clean up go_tls.h section
+ 2021-01-25 54b251f lib/time, time/tzdata: update tzdata to 2021a
+ 2021-01-25 ff82cc9 os: force consistent mtime before running fstest on directory on Windows
+ 2021-01-25 044f937 doc/go1.16: fix WalkDir and Walk links
+ 2021-01-23 b634f5d doc/go1.16: add crypto/x509 memory optimization
+ 2021-01-23 9897655 doc/go1.16: reword ambiguously parsable sentence
+ 2021-01-23 cd99385 cmd/internal/obj/arm64: fix VMOVQ instruction encoding error
+ 2021-01-23 66ee8b1 runtime: restore cgo_import_dynamic for libc.so on openbsd
+ 2021-01-22 25c39e4 io/ioutil: fix example test for WriteFile to allow it to run in the playground
+ 2021-01-22 eb21b31 runtime: define dummy msanmove
+ 2021-01-22 3a778ff runtime: check for g0 stack last in signal handler
+ 2021-01-22 a2cef9b cmd/go: don't lookup the path for CC when invoking cgo

Change-Id: I651949f9eb18b57e3c996c4f3b2b3bf458bc5d97
  • Loading branch information
@mdempsky
mdempsky committed Jan 26, 2021
2 parents 7eaaf28 + bf0f7c9 commit 5e4a0cd
Show file tree
Hide file tree
Showing 22 changed files with 7,400 additions and 7,040 deletions.
72 changes: 58 additions & 14 deletions doc/asm.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -464,6 +464,57 @@ <h3 id="directives">Directives</h3>
</li>
</ul>

<h3 id="data-offsets">Interacting with Go types and constants</h3>

<p>
If a package has any .s files, then <code>go build</code> will direct
the compiler to emit a special header called <code>go_asm.h</code>,
which the .s files can then <code>#include</code>.
The file contains symbolic <code>#define</code> constants for the
offsets of Go struct fields, the sizes of Go struct types, and most
Go <code>const</code> declarations defined in the current package.
Go assembly should avoid making assumptions about the layout of Go
types and instead use these constants.
This improves the readability of assembly code, and keeps it robust to
changes in data layout either in the Go type definitions or in the
layout rules used by the Go compiler.
</p>

<p>
Constants are of the form <code>const_<i>name</i></code>.
For example, given the Go declaration <code>const bufSize =
1024</code>, assembly code can refer to the value of this constant
as <code>const_bufSize</code>.
</p>

<p>
Field offsets are of the form <code><i>type</i>_<i>field</i></code>.
Struct sizes are of the form <code><i>type</i>__size</code>.
For example, consider the following Go definition:
</p>

<pre>
type reader struct {
buf [bufSize]byte
r int
}
</pre>

<p>
Assembly can refer to the size of this struct
as <code>reader__size</code> and the offsets of the two fields
as <code>reader_buf</code> and <code>reader_r</code>.
Hence, if register <code>R1</code> contains a pointer to
a <code>reader</code>, assembly can reference the <code>r</code> field
as <code>reader_r(R1)</code>.
</p>

<p>
If any of these <code>#define</code> names are ambiguous (for example,
a struct with a <code>_size</code> field), <code>#include
"go_asm.h"</code> will fail with a "redefinition of macro" error.
</p>

<h3 id="runtime">Runtime Coordination</h3>

<p>
Expand Down Expand Up @@ -615,21 +666,15 @@ <h3 id="x86">32-bit Intel 386</h3>
<p>
The runtime pointer to the <code>g</code> structure is maintained
through the value of an otherwise unused (as far as Go is concerned) register in the MMU.
An OS-dependent macro <code>get_tls</code> is defined for the assembler if the source is
in the <code>runtime</code> package and includes a special header, <code>go_tls.h</code>:
In the runtime package, assembly code can include <code>go_tls.h</code>, which defines
an OS- and architecture-dependent macro <code>get_tls</code> for accessing this register.
The <code>get_tls</code> macro takes one argument, which is the register to load the
<code>g</code> pointer into.
</p>

<pre>
#include "go_tls.h"
</pre>

<p>
Within the runtime, the <code>get_tls</code> macro loads its argument register
with a pointer to the <code>g</code> pointer, and the <code>g</code> struct
contains the <code>m</code> pointer.
There's another special header containing the offsets for each
element of <code>g</code>, called <code>go_asm.h</code>.
The sequence to load <code>g</code> and <code>m</code> using <code>CX</code> looks like this:
For example, the sequence to load <code>g</code> and <code>m</code>
using <code>CX</code> looks like this:
</p>

<pre>
Expand All @@ -642,8 +687,7 @@ <h3 id="x86">32-bit Intel 386</h3>
</pre>

<p>
Note: The code above works only in the <code>runtime</code> package, while <code>go_tls.h</code> also
applies to <a href="#arm">arm</a>, <a href="#amd64">amd64</a> and amd64p32, and <code>go_asm.h</code> applies to all architectures.
The <code>get_tls</code> macro is also defined on <a href="#amd64">amd64</a>.
</p>

<p>
Expand Down
163 changes: 157 additions & 6 deletions doc/go1.16.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -461,10 +461,10 @@ <h3 id="fs">File Systems</h3>

<p>
The new <a href="/pkg/io/fs/"><code>io/fs</code></a> package
defines an abstraction for read-only trees of files,
the <a href="/pkg/io/fs/#FS"><code>fs.FS</code></a> interface,
and the standard library packages have
been adapted to make use of the interface as appropriate.
defines the <a href="/pkg/io/fs/#FS"><code>fs.FS</code></a> interface,
an abstraction for read-only trees of files.
The standard library packages have been adapted to make use
of the interface as appropriate.
</p>

<p>
Expand Down Expand Up @@ -499,6 +499,44 @@ <h3 id="fs">File Systems</h3>
implementations.
</p>

<h3 id="ioutil">Deprecation of io/ioutil</h3>

<p>
The <a href="/pkg/io/ioutil/"><code>io/ioutil</code></a> package has
turned out to be a poorly defined and hard to understand collection
of things. All functionality provided by the package has been moved
to other packages. The <code>io/ioutil</code> package remains and
will continue to work as before, but we encourage new code to use
the new definitions in the <a href="/pkg/io/"><code>io</code></a> and
<a href="/pkg/os/"><code>os</code></a> packages.

Here is a list of the new locations of the names exported
by <code>io/ioutil</code>:
<ul>
<li><a href="/pkg/io/ioutil/#Discard"><code>Discard</code></a>
=> <a href="/pkg/io/#Discard"><code>io.Discard</code></a></li>
<li><a href="/pkg/io/ioutil/#NopCloser"><code>NopCloser</code></a>
=> <a href="/pkg/io/#NopCloser"><code>io.NopCloser</code></a></li>
<li><a href="/pkg/io/ioutil/#ReadAll"><code>ReadAll</code></a>
=> <a href="/pkg/io/#ReadAll"><code>io.ReadAll</code></a></li>
<li><a href="/pkg/io/ioutil/#ReadDir"><code>ReadDir</code></a>
=> <a href="/pkg/os/#ReadDir"><code>os.ReadDir</code></a>
(note: returns a slice of
<a href="/pkg/os/#DirEntry"><code>os.DirEntry</code></a>
rather than a slice of
<a href="/pkg/fs/#FileInfo"><code>fs.FileInfo</code></a>)
</li>
<li><a href="/pkg/io/ioutil/#ReadFile"><code>ReadFile</code></a>
=> <a href="/pkg/os/#ReadFile"><code>os.ReadFile</code></a></li>
<li><a href="/pkg/io/ioutil/#TempDir"><code>TempDir</code></a>
=> <a href="/pkg/os/#MkdirTemp"><code>os.MkdirTemp</code></a></li>
<li><a href="/pkg/io/ioutil/#TempFile"><code>TempFile</code></a>
=> <a href="/pkg/os/#CreateTemp"><code>os.CreateTemp</code></a></li>
<li><a href="/pkg/io/ioutil/#WriteFile"><code>WriteFile</code></a>
=> <a href="/pkg/os/#WriteFile"><code>os.WriteFile</code></a></li>
</ul>
</p>

<!-- okay-after-beta1
TODO: decide if any additional changes are worth factoring out from
"Minor changes to the library" and highlighting in "Core library"
Expand Down Expand Up @@ -623,6 +661,14 @@ <h3 id="minor_library_changes">Minor changes to the library</h3>
method allows accessing the <a href="/pkg/crypto/x509/#SystemRootsError.Err"><code>Err</code></a>
field through the <a href="/pkg/errors"><code>errors</code></a> package functions.
</p>

<p><!-- CL 230025 -->
On Unix systems, the <code>crypto/x509</code> package is now more
efficient in how it stores its copy of the system cert pool.
Programs that use only a small number of roots will use around a
half megabyte less memory.
</p>

</dd>
</dl><!-- crypto/x509 -->

Expand Down Expand Up @@ -685,6 +731,37 @@ <h3 id="minor_library_changes">Minor changes to the library</h3>
</dd>
</dl><!-- flag -->

<dl id="go/build"><dt><a href="/pkg/go/build/">go/build</a></dt>
<dd>
<p><!-- CL 243941, CL 283636 -->
The <a href="/pkg/go/build/#Package"><code>Package</code></a>
struct has new fields that report information
about <code>//go:embed</code> directives in the package:
<a href="/pkg/go/build/#Package.EmbedPatterns"><code>EmbedPatterns</code></a>,
<a href="/pkg/go/build/#Package.EmbedPatternPos"><code>EmbedPatternPos</code></a>,
<a href="/pkg/go/build/#Package.TestEmbedPatterns"><code>TestEmbedPatterns</code></a>,
<a href="/pkg/go/build/#Package.TestEmbedPatternPos"><code>TestEmbedPatternPos</code></a>,
<a href="/pkg/go/build/#Package.XTestEmbedPatterns"><code>XTestEmbedPatterns</code></a>,
<a href="/pkg/go/build/#Package.XTestEmbedPatternPos"><code>XTestEmbedPatternPos</code></a>.
</p>

<p><!-- CL 240551 -->
The <a href="/pkg/go/build/#Package"><code>Package</code></a> field
<a href="/pkg/go/build/#Package.IgnoredGoFiles"><code>IgnoredGoFiles</code></a>
will no longer include files that start with "_" or ".",
as those files are always ignored.
<code>IgnoredGoFiles</code> is for files ignored because of
build constraints.
</p>

<p><!-- CL 240551 -->
The new <a href="/pkg/go/build/#Package"><code>Package</code></a>
field <a href="/pkg/go/build/#Package.IgnoredOtherFiles"><code>IgnoredOtherFiles</code></a>
has a list of non-Go files ignored because of build constraints.
</p>
</dd>
</dl><!-- go/build -->

<dl id="html/template"><dt><a href="/pkg/html/template/">html/template</a></dt>
<dd>
<p><!-- CL 243938 -->
Expand All @@ -703,6 +780,15 @@ <h3 id="minor_library_changes">Minor changes to the library</h3>
The package now defines a
<a href="/pkg/io/#ReadSeekCloser"><code>ReadSeekCloser</code></a> interface.
</p>

<p><!-- CL 263141 -->
The package now defines
<a href="/pkg/io/#Discard"><code>Discard</code></a>,
<a href="/pkg/io/#NopCloser"><code>NopCloser</code></a>, and
<a href="/pkg/io/#ReadAll"><code>ReadAll</code></a>,
to be used instead of the same names in the
<a href="/pkg/io/ioutil/"><code>io/ioutil</code></a> package.
</p>
</dd>
</dl><!-- io -->

Expand Down Expand Up @@ -857,6 +943,52 @@ <h3 id="minor_library_changes">Minor changes to the library</h3>
instead of the unexported <code>errFinished</code> when the process has
already finished.
</p>

<p><!-- CL 261540 -->
The package defines a new type
<a href="/pkg/os/#DirEntry"><code>DirEntry</code></a>
as an alias for <a href="/pkg/io/fs/#DirEntry"><code>fs.DirEntry</code></a>.
The new <a href="/pkg/os/#ReadDir"><code>ReadDir</code></a>
function and the new
<a href="/pkg/os/#File.ReadDir"><code>File.ReadDir</code></a>
method can be used to read the contents of a directory into a
slice of <a href="/pkg/os/#DirEntry"><code>DirEntry</code></a>.
The <a href="/pkg/os/#File.Readdir"><code>File.Readdir</code></a>
method (note the lower case <code>d</code> in <code>dir</code>)
still exists, returning a slice of
<a href="/pkg/os/#FileInfo"><code>FileInfo</code></a>, but for
most programs it will be more efficient to switch to
<a href="/pkg/os/#File.ReadDir"><code>File.ReadDir</code></a>.
</p>

<p><!-- CL 263141 -->
The package now defines
<a href="/pkg/os/#CreateTemp"><code>CreateTemp</code></a>,
<a href="/pkg/os/#MkdirTemp"><code>MkdirTemp</code></a>,
<a href="/pkg/os/#ReadFile"><code>ReadFile</code></a>, and
<a href="/pkg/os/#WriteFile"><code>WriteFile</code></a>,
to be used instead of functions defined in the
<a href="/pkg/io/ioutil/"><code>io/ioutil</code></a> package.
</p>

<p><!-- CL 243906 -->
The types <a href="/pkg/os/#FileInfo"><code>FileInfo</code></a>,
<a href="/pkg/os/#FileMode"><code>FileMode</code></a>, and
<a href="/pkg/os/#PathError"><code>PathError</code></a>
are now aliases for types of the same name in the
<a href="/pkg/io/fs/"><code>io/fs</code></a> package.
Function signatures in the <a href="/pkg/os/"><code>os</code></a>
package have been updated to refer to the names in the
<a href="/pkg/io/fs/"><code>io/fs</code></a> package.
This should not affect any existing code.
</p>

<p><!-- CL 243911 -->
The new <a href="/pkg/os/#DirFS"><code>DirFS</code></a> function
provides an implementation of
<a href="/pkg/io/fs/#FS"><code>fs.FS</code></a> backed by a tree
of operating system files.
</p>
</dd>
</dl><!-- os -->

Expand Down Expand Up @@ -887,9 +1019,9 @@ <h3 id="minor_library_changes">Minor changes to the library</h3>
<dd>
<p><!-- CL 267887 -->
The new function
<a href="/pkg/path/filepath/WalkDir"><code>WalkDir</code></a>
<a href="/pkg/path/filepath/#WalkDir"><code>WalkDir</code></a>
is similar to
<a href="/pkg/path/filepath/Walk"><code>Walk</code></a>,
<a href="/pkg/path/filepath/#Walk"><code>Walk</code></a>,
but is typically more efficient.
The function passed to <code>WalkDir</code> receives a
<a href="/pkg/io/fs/#DirEntry"><code>fs.DirEntry</code></a>
Expand Down Expand Up @@ -975,6 +1107,25 @@ <h3 id="minor_library_changes">Minor changes to the library</h3>
</dd>
</dl><!-- syscall -->

<dl id="testing/iotest"><dt><a href="/pkg/testing/iotest/">testing/iotest</a></dt>
<dd>
<p><!-- CL 199501 -->
The new
<a href="/pkg/testing/iotest/#ErrReader"><code>ErrReader</code></a>
function returns an
<a href="/pkg/io/#Reader"><code>io.Reader</code></a> that always
returns an error.
</p>

<p><!-- CL 243909 -->
The new
<a href="/pkg/testing/iotest/#TestReader"><code>TestReader</code></a>
function tests that an <a href="/pkg/io/#Reader"><code>io.Reader</code></a>
behaves correctly.
</p>
</dd>
</dl><!-- testing/iotest -->

<dl id="text/template"><dt><a href="/pkg/text/template/">text/template</a></dt>
<dd>
<p><!-- CL 254257, golang.org/issue/29770 -->
Expand Down
4 changes: 2 additions & 2 deletions lib/time/update.bash
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@
# Consult https://www.iana.org/time-zones for the latest versions.

# Versions to use.
CODE=2020f
DATA=2020f
CODE=2021a
DATA=2021a

set -e
rm -rf work
Expand Down
Binary file modified lib/time/zoneinfo.zip
View file
Binary file not shown.
40 changes: 31 additions & 9 deletions src/cmd/compile/internal/walk/assign.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -289,11 +289,14 @@ func ascompatee(op ir.Op, nl, nr []ir.Node) []ir.Node {
}

var assigned ir.NameSet
var memWrite bool
var memWrite, deferResultWrite bool

// affected reports whether expression n could be affected by
// the assignments applied so far.
affected := func(n ir.Node) bool {
if deferResultWrite {
return true
}
return ir.Any(n, func(n ir.Node) bool {
if n.Op() == ir.ONAME && assigned.Has(n.(*ir.Name)) {
return true
Expand Down Expand Up @@ -369,21 +372,40 @@ func ascompatee(op ir.Op, nl, nr []ir.Node) []ir.Node {

appendWalkStmt(&late, convas(ir.NewAssignStmt(base.Pos, lorig, r), &late))

if name != nil && ir.IsBlank(name) {
// We can ignore assignments to blank.
// Check for reasons why we may need to compute later expressions
// before this assignment happens.

if name == nil {
// Not a direct assignment to a declared variable.
// Conservatively assume any memory access might alias.
memWrite = true
continue
}
if op == ir.ORETURN && types.OrigSym(name.Sym()) == nil {
// We can also ignore assignments to anonymous result
// parameters. These can't appear in expressions anyway.

if name.Class == ir.PPARAMOUT && ir.CurFunc.HasDefer() {
// Assignments to a result parameter in a function with defers
// becomes visible early if evaluation of any later expression
// panics (#43835).
deferResultWrite = true
continue
}

if name != nil && name.OnStack() && !name.Addrtaken() {
assigned.Add(name)
} else {
if sym := types.OrigSym(name.Sym()); sym == nil || sym.IsBlank() {
// We can ignore assignments to blank or anonymous result parameters.
// These can't appear in expressions anyway.
continue
}

if name.Addrtaken() || !name.OnStack() {
// Global variable, heap escaped, or just addrtaken.
// Conservatively assume any memory access might alias.
memWrite = true
continue
}

// Local, non-addrtaken variable.
// Assignments can only alias with direct uses of this variable.
assigned.Add(name)
}

early.Append(late.Take()...)
Expand Down
Loading

0 comments on commit 5e4a0cd

Please sign in to comment.