Update license tool to check several additional directories

Some of these directories are currently excluded from the license check. This change fixes the metadata where required and removes them from PRUNE_PATHS.

Other directories contain third-party code, but aren't in directories named 'third-party', so aren't checked by the license tool by default. This change fixes the metadata where required and adds them to ADDITIONAL_PATHS.

To avoid adding to about:credits licenses for modules that don't ship as part of the product, this change also introduces a special 'NOT_SHIPPED' value for the 'License File' field.

This will simplify the process of adding a tool to check license compatibility for the Android WebView build.

BUG=138921

Review URL: https://chromiumcodereview.appspot.com/10830201

git-svn-id: svn://svn.chromium.org/chrome/trunk/src@150800 0039d316-1c4b-4281-b951-d872f2087c98

base/third_party/icu/README.chromium

@@ -1,10 +1,15 @@
 Name: ICU
 URL: http://site.icu-project.org/
+License: MIT
+License File: NOT_SHIPPED
 
 This file has the relevant components from ICU copied to handle basic
 UTF8/16/32 conversions. Components are copied from utf.h utf8.h utf16.h and
 utf_impl.c
 
+The same module appears in third_party/icu, so we don't repeat the license
+file here.
+
 The main change is that U_/U8_/U16_ prefixes have been replaced with
 CBU_/CBU8_/CBU16_ (for "Chrome Base") to avoid confusion with the "real" ICU
 macros should ICU be in use on the system. For the same reason, the functions

breakpad/README.chromium

@@ -1,7 +1,7 @@
 Name: Breakpad, An open-source multi-platform crash reporting system
 Short Name: breakpad
 URL: http://code.google.com/p/google-breakpad/
-License: New BSD, plus MIT/X derivate for libcurl
+License: New BSD, MIT/X derivate, Apple PSL 2.0 and Apache 2.0 licenses
 Security Critical: no
 
 This directory contains Chrome's version of Breakpad's Visual Studio build

chrome/common/extensions/docs/examples/README.chromium

@@ -0,0 +1,5 @@
+Name: Extensions examples
+URL: None
+License: Apache 2.0, MIT, X11, BSD, and GPL v2 licenses
+License File: NOT_SHIPPED
+Security Critical: no

chrome/test/chromeos/autotest/README.chromium

@@ -0,0 +1,5 @@
+Name: Test code
+URL: unknown
+License: BSD
+License File: /LICENSE.chromium_os
+Security Critical: no

chrome/test/data/README.chromium

@@ -0,0 +1,5 @@
+Name: Test data
+URL: None
+License: Many, including (L)GPL v3
+License File: NOT_SHIPPED
+Security Critical: no

net/third_party/mozilla_security_manager/README.chromium

@@ -2,10 +2,15 @@ Name: Mozilla Personal Security Manager
 URL: http://mxr.mozilla.org/mozilla1.9.2/source/security/manager/
 InfoURL: http://www.mozilla.org/
 Version: Mozilla 1.9.2
+License: MPL 1.1 / GPL 2.0 / LGPL 2.1
+License File: NOT_SHIPPED
 
 Description:
 This is selected code bits from Mozilla's Personal Security Manager.
 
+The same module appears in chrome/third_party/mozilla_security_manager, so we
+don't repeat the license file here.
+
 Local Modifications:
 Files are forked from Mozilla's because of the heavy adaptations necessary.
 Differences are using Chromium localization and other libraries instead of

net/third_party/nss/README.chromium

@@ -2,10 +2,15 @@ Name: Network Security Services (NSS)
 URL: http://www.mozilla.org/projects/security/pki/nss/
 Version: 3.13.4 pre-release snapshot 20120319
 Security Critical: Yes
+License: MPL 1.1/GPL 2.0/LGPL 2.1
+License FILE: NOT_SHIPPED
 
 This directory includes a copy of NSS's libssl from the CVS repo at:
   :pserver:anonymous@cvs-mirror.mozilla.org:/cvsroot
 
+The same module appears in crypto/third_party/nss (and third_party/nss on some
+platforms), so we don't repeat the license file here.
+
 The snapshot was updated to the CVS tag: NSS_SSL_3_13_4_20120319_TAG
 
 Patches:

net/tools/spdyshark/README.chromium

@@ -0,0 +1,4 @@
+Name: Spdyshark
+URL: http://anonsvn.wireshark.org/wireshark/trunk
+License: GPL v2
+Security Critical: no

third_party/README.chromium.template

@@ -5,7 +5,7 @@ Version: A searchable version number for the package (if the package does not ve
 Date: (OPTIONAL if version is supplied) The date that the package was updated
 Revision: (OPTIONAL if version is supplied) The current revision of the package
 License: The license under which the package is distributed. Arbitrary text is allowed, but prefer standard forms, eg MIT/X11/BSD/Apache 2.0/GPL/LGPL.
-License File: (OPTIONAL) File that contains a copy of the packages license
+License File: (OPTIONAL) File that contains a copy of the package's license. Use the special value NOT_SHIPPED to indicate that the package is not included in the shipped product, so its license does not need to be included in about:credits and no license file is required.
 Security Critical: Either yes or no depending on whether this package is shipped in releases. For example openssl is critical where cygwin is not.
 
 Description:

third_party/harfbuzz-ng/README.chromium

@@ -5,6 +5,8 @@ Version: unknown
 Date: 20120608
 Revision: 0bb0f5d41976ae27c5c7a51cbb82144b48315a4b
 Security Critical: yes
+License: MIT
+License File: NOT_SHIPPED
 
 Description:
 This is harfbuzz-ng, a new implementation of harfbuzz with a different

tools/licenses.py

@@ -21,34 +21,16 @@
 
 # Paths from the root of the tree to directories to skip.
 PRUNE_PATHS = set([
-    # Same module occurs in both the top-level third_party and others.
-    os.path.join('base','third_party','icu'),
-
-    # Assume for now that breakpad has their licensing in order.
-    os.path.join('breakpad'),
-
-    # Assume for now that native client has their licensing in order.
-    os.path.join('native_client'),
-
-    # Same module occurs in chrome/ and in net/, so skip one of them.
-    os.path.join('net','third_party','mozilla_security_manager'),
-
-    # Same module occurs in base/, net/, and src/ so skip all but one of them.
+    # Same module occurs in crypto/third_party/nss and net/third_party/nss, so
+    # skip this one.
     os.path.join('third_party','nss'),
-    os.path.join('net','third_party','nss'),
 
     # Placeholder directory only, not third-party code.
     os.path.join('third_party','adobe'),
 
-    # Same license as Chromium.
-    os.path.join('third_party','lss'),
-
     # Only binaries, used during development.
     os.path.join('third_party','valgrind'),
 
-    # Directories that are the same as those in base/third_party.
-    os.path.join('v8','src','third_party','valgrind'),
-
     # Used for development and test, not in the shipping product.
     os.path.join('third_party','bidichecker'),
     os.path.join('third_party','cygwin'),
@@ -70,9 +52,6 @@
     os.path.join('third_party','directxsdk'),
     os.path.join('third_party','platformsdk_win2008_6_1'),
     os.path.join('third_party','platformsdk_win7'),
-
-    # Harfbuzz-ng is not currently shipping in any product:
-    os.path.join('third_party','harfbuzz-ng'),
 ])
 
 # Directories we don't scan through.
@@ -81,12 +60,26 @@
               'layout_tests')             # lots of subdirs
 
 ADDITIONAL_PATHS = (
+    os.path.join('breakpad'),
+    os.path.join('chrome', 'common', 'extensions', 'docs', 'examples'),
+    os.path.join('chrome', 'test', 'chromeos', 'autotest'),
+    os.path.join('chrome', 'test', 'data'),
     os.path.join('googleurl'),
+    os.path.join('native_client'),
     os.path.join('native_client_sdk'),
+    os.path.join('net', 'tools', 'spdyshark'),
     os.path.join('ppapi'),
+    os.path.join('sandbox', 'linux', 'seccomp-legacy'),
+    os.path.join('sdch', 'open-vcdiff'),
+    os.path.join('testing', 'gmock'),
+    os.path.join('testing', 'gtest'),
     # The directory with the word list for Chinese and Japanese segmentation
     # with different license terms than ICU.
     os.path.join('third_party','icu','source','data','brkitr'),
+    os.path.join('tools', 'grit'),
+    os.path.join('tools', 'gyp'),
+    os.path.join('tools', 'page_cycler', 'acid3'),
+    os.path.join('v8'),
     # Fake directory so we can include the strongtalk license.
     os.path.join('v8', 'strongtalk'),
 )
@@ -96,12 +89,37 @@
 # can't provide a README.chromium.  Please prefer a README.chromium
 # wherever possible.
 SPECIAL_CASES = {
-    'googleurl': {
+    os.path.join('googleurl'): {
         "Name": "google-url",
         "URL": "http://code.google.com/p/google-url/",
         "License": "BSD and MPL 1.1/GPL 2.0/LGPL 2.1",
         "License File": "LICENSE.txt",
     },
+    os.path.join('native_client'): {
+        "Name": "native client",
+        "URL": "http://code.google.com/p/nativeclient",
+        "License": "BSD",
+    },
+    os.path.join('sandbox', 'linux', 'seccomp-legacy'): {
+        "Name": "seccompsandbox",
+        "URL": "http://code.google.com/p/seccompsandbox",
+        "License": "BSD",
+    },
+    os.path.join('sdch', 'open-vcdiff'): {
+        "Name": "open-vcdiff",
+        "URL": "http://code.google.com/p/open-vcdiff",
+        "License": "Apache 2.0, MIT, GPL v2 and custom licenses",
+    },
+    os.path.join('testing', 'gmock'): {
+        "Name": "gmock",
+        "URL": "http://code.google.com/p/googlemock",
+        "License": "BSD",
+    },
+    os.path.join('testing', 'gtest'): {
+        "Name": "gtest",
+        "URL": "http://code.google.com/p/googletest",
+        "License": "BSD",
+    },
     os.path.join('third_party', 'angle'): {
         "Name": "Almost Native Graphics Layer Engine",
         "URL": "http://code.google.com/p/angleproject/",
@@ -123,6 +141,8 @@
     os.path.join('third_party', 'lss'): {
         "Name": "linux-syscall-support",
         "URL": "http://code.google.com/p/lss/",
+        "License": "BSD",
+        "License File": "/LICENSE",
     },
     os.path.join('third_party', 'ots'): {
         "Name": "OTS (OpenType Sanitizer)",
@@ -166,6 +186,21 @@
         "URL": "http://code.google.com/p/web-page-replay",
         "License": "Apache 2.0",
     },
+    os.path.join('tools', 'grit'): {
+        "Name": "grit",
+        "URL": "http://code.google.com/p/grit-i18n",
+        "License": "BSD",
+    },
+    os.path.join('tools', 'gyp'): {
+        "Name": "gyp",
+        "URL": "http://code.google.com/p/gyp",
+        "License": "BSD",
+    },
+    os.path.join('v8'): {
+        "Name": "gyp",
+        "URL": "http://code.google.com/p/v8",
+        "License": "BSD",
+    },
     os.path.join('v8', 'strongtalk'): {
         "Name": "Strongtalk",
         "URL": "http://www.strongtalk.org/",
@@ -175,6 +210,11 @@
     },
 }
 
+# Special value for 'License File' field used to indicate that the license file
+# should not be used in about:credits.
+NOT_SHIPPED = "NOT_SHIPPED"
+
+
 class LicenseError(Exception):
     """We raise this exception when a directory's licensing info isn't
     fully filled out."""
@@ -234,19 +274,22 @@ def ParseDir(path):
                                "in README.chromium or licences.py "
                                "SPECIAL_CASES")
 
-    # Check that the license file exists.
-    for filename in (metadata["License File"], "COPYING"):
-        license_path = AbsolutePath(path, filename)
-        if license_path is not None:
-            metadata["License File"] = license_path
-            break
+    # Special-case modules that aren't in the shipping product, so don't need
+    # their license in about:credits.
+    if metadata["License File"] != NOT_SHIPPED:
+        # Check that the license file exists.
+        for filename in (metadata["License File"], "COPYING"):
+            license_path = AbsolutePath(path, filename)
+            if license_path is not None:
+                metadata["License File"] = license_path
+                break
 
-    if not license_path:
-        raise LicenseError("License file not found. "
-                           "Either add a file named LICENSE, "
-                           "import upstream's COPYING if available, "
-                           "or add a 'License File:' line to README.chromium "
-                           "with the appropriate path.")
+        if not license_path:
+            raise LicenseError("License file not found. "
+                               "Either add a file named LICENSE, "
+                               "import upstream's COPYING if available, "
+                               "or add a 'License File:' line to "
+                               "README.chromium with the appropriate path.")
 
     if "Required Text" in metadata:
         required_path = AbsolutePath(path, metadata["Required Text"])
@@ -350,6 +393,10 @@ def EvaluateTemplate(template, env, escape=True):
             print >>sys.stderr, ("WARNING: licensing info for " + path +
                                  " is incomplete, skipping.")
             continue
+        if metadata['License File'] == NOT_SHIPPED:
+            print >>sys.stderr, ("Path " + path + " marked as " + NOT_SHIPPED +
+                                 ", skipping.")
+            continue
         env = {
             'name': metadata['Name'],
             'url': metadata['URL'],