cdefs.h: Add back comment about branch prediction

Add back, with editing, the comments about branch prediction, when to
use it, etc. Offer stronger opinions about this in style(9). Add in the
convention for FreeBSD that we do only the entire expression in the if
expression. Advise use only when it makes things measurably faster.

Requested by:		jhb
Sponsored by:		Netflix
Reviewed by:		brooks, jhb
Differential Revision:	https://reviews.freebsd.org/D45837

share/man/man9/style.9

@@ -897,6 +897,22 @@ New code should use
 .Fn _Static_assert
 instead of the older
 .Fn CTASSERT .
+.Pp
+.Fn __predict_true
+and
+.Fn __predict_false
+should only be used in frequently executed code when it makes the code
+measurably faster.
+It is wasteful to make predictions for infrequently run code, like subsystem
+initialization.
+When using branch prediction hints, atypical error conditions should use
+.Fn __predict_false
+(document the exceptions).
+Operations that almost always succeed use
+.Fn __predict_true .
+Only use the annotation for the entire if statement, rather than individual clauses.
+Do not add these annotations without empirical evidence of the likelihood of the
+branch.
 .Sh FILES
 .Bl -tag -width indent
 .It Pa /usr/src/tools/build/checkstyle9.pl

sys/sys/cdefs.h

@@ -321,6 +321,13 @@
 #define	__restrict	restrict
 #endif
 
+/*
+ * All modern compilers have explicit branch prediction so that the CPU back-end
+ * can hint to the processor and also so that code blocks can be reordered such
+ * that the predicted path sees a more linear flow, thus improving cache
+ * behavior, etc. Use sparingly, except in performance critical code where
+ * they make things measurably faster.
+ */
 #define	__predict_true(exp)     __builtin_expect((exp), 1)
 #define	__predict_false(exp)    __builtin_expect((exp), 0)