-
Notifications
You must be signed in to change notification settings - Fork 3
List of useful sites for monitoring DUNE data
https://github.com/DUNE/FAQ/projects/1
https://github.com/DUNE/dune-prod-utils/issues
https://github.com/DUNE/data-mgmt-ops/issues
https://op-webtools.web.cern.ch/vistar/?usr=SPS1
SPS schedule https://ps-sps-coordination.web.cern.ch/ps-sps-coordination/
Master page for np02/np04 daq https://twiki.cern.ch/twiki/bin/view/CENF/DUNEDAQProto
(Note: see below for how to set up proxies that allow external access to CERN sites.)
NP04 ELISA logbook https://pdsp-elog.app.cern.ch/elisa/display (requires CERN login and membership in an E-group.)
NP02-coldbox logbook https://np-vd-coldbox-elog.app.cern.ch/elisa/display?logbook=np-vd-coldbox-elog (requires CERN login)
Overview of the CERN FTS3 instance:
https://fts3-public.cern.ch:8449/fts3/ftsmon/#/
Grafana for the CERN FTS3 instance: (needs CERN SSO)
https://monit-grafana.cern.ch/d/veRQSWBGz/fts-servers-dashboard
Grafana for just DUNE on CERN FTS3: (needs CERN SSO) https://monit-grafana.cern.ch/d/veRQSWBGz/fts-servers-dashboard?orgId=25&var-group_by=vo&var-vo=dune&var-src_country=All&var-dst_country=All&var-src_site=All&var-source_se=All&var-dst_site=All&var-dest_se=All&var-fts_server=All&var-staging=All&var-bin=1h&var-dest_hostname=All&var-source_hostname=All
IngestD: https://dune.monitoring.edi.scotgrid.ac.uk/app/dashboards#/view/28002070-2f54-11ef-b9ca-6fc4e3f16d60
DeclaD: https://dune.monitoring.edi.scotgrid.ac.uk/app/dashboards#/view/c97984b0-2f53-11ef-b9ca-6fc4e3f16d60
MetaCat demo:
DUNE MetaCat GUI: https://metacat.fnal.gov:9443/dune_meta_demo/app/gui/index METACAT_AUTH_SERVER_URL=https://metacat.fnal.gov:8143/auth/dune_dev METACAT_SERVER_URL=https://metacat.fnal.gov:9443/dune_meta_prod/app
MetaCat production:
DUNE MetaCat GUI: https://metacat.fnal.gov:9443/dune_meta_prod/app/gui/index
METACAT_SERVER_URL=https://metacat.fnal.gov:9443/dune_meta_prod/app
METACAT_AUTH_SERVER_URL=https://metacat.fnal.gov:8143/auth/dune
DUNE MetaCat GUI: https://metacat.fnal.gov:9443/dune_meta_prod/app/gui/datasets
MetaCat history:
https://dbdata0vm.fnal.gov:8043/service_charts/dune-meta-prod?window=week (SSO)
IFBEAM https://dbweb8.fnal.gov:8443/ifbeam/bmon/pdunemon/Display
Run config: NEED TO UPDATE WITH CURRENT WEBSITE
Overview talk from database meeting: https://indico.fnal.gov/event/54126/contributions/239184/attachments/154057/200060/slow-cotrols-data_lino_gerlach.pdf
Limited access: detector experts access that data via a website https://np04-slow-control.web.cern.ch/
DCS-DB is accessed via read-only rest API https://gitlab.cern.ch/ep-dt-di/dcsdb-rest
DB itself can only be accessed from within CERN network and CERN strongly discourages the use of vpns for this purpose
THIS LINK IS DEAD AND THAT REPO DOES NOT EXIST NOW https://gitlab.cern.ch/ep-dt-di/dcsdb-rest gave me a 404 error after CERN authentication. -> only a few members have been granted access to that repo -> I contacted one of the maintainers Maybe I need to join a group? We may not need this but having some insight is useful.
FNAL systems status: https://fermi.servicenowservices.com/wp?id=evg-service-status (requires SSO)
DUNE CRIC - https://dune-cric.cern.ch/core/experimentsite/list/
EGI https://accounting.egi.eu/wlcg/countries/
THIS LINK IS BUSTED OSU global map http://fermicloud584.fnal.gov:3000/new under construction
Perfsonar: https://psmad.opensciencegrid.org/maddash-webui/index.cgi?dashboard=DUNE%20Mesh%20Config
SAM project monitoring https://samweb.fnal.gov:8483/station_monitor/dune/stations/dune/projects
Data Dispatcher project monitoring https://metacat.fnal.gov:9443/dune/dd/gui/P/projects
Enstore monitoring https://www-stken.fnal.gov/enstore/encp_enstore_system.html
Dcache network https://mrtg.fnal.gov/weathermap/wx-protodune.html (requires proxy or VPN)
ESNET plot https://my.es.net/oscars/view/RR94 (changed from the previous /P6FG)
DUNE FTS monitoring: https://dune.monitoring.edi.scotgrid.ac.uk/app/dashboards#/view/c9268890-749c-11ee-a635-c11ccc9f77dd
protoDUNE data pipeline monitoring: https://dune.monitoring.edi.scotgrid.ac.uk/app/dashboards#/view/21a10bf0-0226-11ef-b9ca-6fc4e3f16d60
DAQ machines storage usage: https://dune.monitoring.edi.scotgrid.ac.uk/app/dashboards#/view/6a95ab70-04da-11ef-b9ca-6fc4e3f16d60
(Heidi’s old link https://wiki.dunescience.org/wiki/Useful_Production_Links)
JustIn https://justin.dune.hep.ac.uk/docs/
JustIn monitoring from JustIn db: https://dune.monitoring.edi.scotgrid.ac.uk/app/dashboards#/view/4717b1e0-175c-11ef-b9ca-6fc4e3f16d60
POMS https://pomsgpvm01.fnal.gov/poms/show_campaigns
Fifebatch - can check out batch sites https://fifemon.fnal.gov/kibana/goto/ee9456df25ce77417eaee9ffa56c4def
offsite running jobs for last 24 hours: https://lsvip.fnal.gov/monitor/d/000000053/experiment-batch-details?orgId=1&var-experiment=dune&viewPanel=10
jobs efficiency for last 24 hours: https://fifemon.fnal.gov/monitor/d/000000069/user-efficiency-details?orgId=1&from=now-7d&to=now-5m&var-user=dunepro
jobs distribution among countries: https://fifemon.fnal.gov/kibana/goto/79b9c2e5e06828809d3a7ea410045bff
Slack channel https://dunescience.slack.com/archives/C6GFAPG4U
Service Now https://fermi.servicenowservices.com/wp
Service now service catalog: (restricted) https://fermi.servicenowservices.com/nav_to.do?uri=%2Flist_service_areas.do%3Fdivision%3Dscientific_computing
github area for issues? DUNE Data Challenge 4
https://github.com/DUNE/dist-comp/labels/Operations
FNAL db map https://dbweb0.fnal.gov/redirector/index.html
Workflow https://dune.github.io/dune-wfs/
Metacat documentation https://dbdata0vm.fnal.gov:9443/dune_meta_demo/app/static/docs/html/index.html
DUNE Computing training https://dune.github.io/computing-training-basics/index.html
Current wiki https://wiki.dunescience.org/wiki/DUNE_Computing (edit this one)
Development wiki https://wiki.dunescience.org/wiki/DUNE_Computing_Wiki_Dev (do not edit, just comment) Accessing websites behind VPN’s
Here is an example of syntax for accessing particular sites at CERN and FNAL
ssh username@lxplus.cern.ch -L 8009:p3s-content.cern.ch:80 -L 8008:p3s-web.cern.ch:80 -L 8080:protodune-ftslight7.cern.ch:8080 -L 8123:np04-srv-001.cern.ch:8123
Simplified without p3s-web (which is now defunct)
Ssh username@lxplus.cern.ch -L 8080:protodune-ftslight7.cern.ch:8080
ssh -l $FNALUSER dunegpvm01.fnal.gov -L 8787:dunesamgpvm02.fnal.gov:80 -L 8788:mrtg.fnal.gov:8788
THIS SHOULD EITHER BE NUKED OR START TO GET USED Logbook for data challenge https://dbweb8.fnal.gov:8443/ECL/dunepro/E/index?category_id=16 ask Heidi to become a member if you cannot create entries.
Old: https://wiki.dunescience.org/wiki/Setting_up_monitoring_pages which pointed to many of the pages.
Need experts to suggest a list of the sites and suggestions on how to get proxies to access. Old page was https://wiki.dunescience.org/wiki/Data_Movement_Web_Pages_-_detailed_documentation but most links are now busted.