[docs] Explain multiclock use and restrictions (#528)

* [docs] Explain multiclock use and restrictions

* Apply suggestions from code review

Co-Authored-By: Albert Magyar <albert.magyar@gmail.com>
Co-Authored-By: alonamid <alonamid@eecs.berkeley.edu>

* Fix doc-induced breakages

Co-authored-by: Albert Magyar <albert.magyar@gmail.com>
Co-authored-by: alonamid <alonamid@eecs.berkeley.edu>

docs/Advanced-Usage/Generating-Different-Targets.rst

@@ -16,13 +16,13 @@ transformed and thus used in FireSim:
 
 #. The top-level module must have no inputs or outputs. Input stimulus and output capture must be 
    implemented using target RTL or target-to-host Bridges.
-#. All target clocks must be generated by single ``RationalClockBridge``.
+#. All target clocks must be generated by a single ``RationalClockBridge``.
 #. Black boxes must be "clock-gateable" by replacing its input clock with a gated equivalent which will be used
    to stall simulation time in that module.
-  #. As a consequence, target clock-gating cannot be implemented using black-box primitives, and must instead be modelled by
-     adding clock-enables to all state elements of the gated clock domain (i.e., by adding a feedback mux on registers to
+
+   a. As a consequence, target clock-gating cannot be implemented using black-box primitives, and must instead be modeled by
+      adding clock-enables to all state elements of the gated clock domain (i.e., by adding an enable or feedback mux on registers to
       conditionally block updates, and by gating write-enables on memories).
-     .
 #. Asynchronous reset must only be implemented using Rocket Chip's black-box async reset. 
    These are replaced with synchronously reset registers using a FIRRTL transformation.
 
@@ -48,6 +48,59 @@ section of the Chipyard documentation.
 #. This allows any Verilog module, subject to the constraint above, to be instantiated anywhere in the target
    design using the standard Chisel Blackbox interface.
 
+----------------------------
+Multiple Clock Domains
+----------------------------
+
+FireSim can support simulating targets that have multiple clock
+domains.  As stated above, all clocks must be generated using a single
+``RationalClockBridge``. For most users the default FireSim test harness in Chipyard will suffice,
+if you need to define a custom test harness instantiate the ``RationalClockBridge`` like so:
+
+.. literalinclude:: ../../sim/src/main/scala/midasexamples/TrivialMulticlock.scala
+    :language: scala
+    :start-after: DOC include start: RationalClockBridge Usage
+    :end-before: DOC include end: RationalClockBridge Usage
+
+
+Further documentation can be found in the source
+(``sim/midas/src/main/scala/midas/widgets/ClockBridge.scala``).
+
+=================
+The Base Clock
+=================
+By convention, target time is specified in cycles of the `base clock`,
+which is defined to be the zeroth output clock of the ``RationalClockBridge``.
+While we suggest making the base clock the fastest clock in your system, as in any
+microprocessor-based system it will likely correspond to your core clock
+frequency, this is not a constraint.
+
+============
+Limitations:
+============
+* The number of target clocks FireSim can simulate is bounded by the number of BUFGCE resources
+  available on the host FPGA, as these are used to independently clock-gate each target clock.
+* As its name suggests, the ``RationalClockBridge`` can only generate target clocks
+  that are rationally related. Specifically, all requested frequencies must be
+  expressable in the form:
+
+  .. math::
+    f_{i} = \frac{f_{lcm}}{k_{i}}
+
+  Where,
+    * :math:`f_{i}` is the desired frequency of the :math:`i^{th}` clock
+    * :math:`f_{lcm}`, is the least-common multiple of all requested frequencies
+    * :math:`k_{i}` is a 16-bit unsigned integer
+
+
+  An arbitrary frequency can be modeled using a sufficiently precise rational
+  multiple. Golden Gate will raise a compile-time error if it cannot support
+  a desired frequency.
+* Each bridge module must reside entirely within a single clock domain. The Bridge's target interface
+  must contain a single input clock, and all inputs and outputs of the
+  bridge module must be latched and launched, respectively, by registers in the
+  same clock domain.
+
 .. _generating-different-targets:
 
 Provided Target Designs

docs/requirements.txt

@@ -1,4 +1,4 @@
-Sphinx==1.7.4
+Sphinx==1.8.5
 Pygments==2.2.0
 sphinx-autobuild
 sphinx_rtd_theme==0.2.5b1

sim/src/main/scala/midasexamples/TrivialMulticlock.scala

@@ -23,11 +23,22 @@ class RegisterModule extends MultiIOModule {
 
 class TrivialMulticlock extends RawModule {
   // TODO: Resolve bug in PeekPoke bridge for 3/7 case
-  //val clockBridge = Module(new RationalClockBridge(1000, (1,2), (1,3), (3,7)))
   //val List(fullRate, halfRate, thirdRate, threeSeventhsRate) = clockBridge.io.clocks.toList
+
+  // DOC include start: RationalClockBridge Usage
+  // Here we request three target clocks (the base clock is implicit). All
+  // clocks beyond the base clock are specified using the RationalClock case
+  // class which gives the clock domain's name, and its clock multiplier and
+  // divisor relative to the base clock.
   val clockBridge = Module(new RationalClockBridge(RationalClock("HalfRate", 1, 2), 
                                                    RationalClock("ThirdRate", 1, 3)))
-  val List(fullRate, halfRate, thirdRate) = clockBridge.io.clocks.toList
+
+  // The clock bridge has a single output: a Vec[Clock] of the requested clocks
+  // in the order they were specified, which we are now free to use through our
+  // Chisel design.  While not necessary, here we unassign the Vec to give them
+  // more informative references in our Chisel.
+  val Seq(fullRate, halfRate, thirdRate) = clockBridge.io.clocks.toSeq
+  // DOC include end: RationalClockBridge Usage
   val reset = WireInit(false.B)
 
   withClockAndReset(fullRate, reset) {
@@ -36,18 +47,18 @@ class TrivialMulticlock extends RawModule {
     val thirdRateInst = Module(new RegisterModule)
     thirdRateInst.slowClock := thirdRate
     thirdRateInst.in := halfRateInst.in
-    val threeSeventhsRateInst = Module(new RegisterModule)
-    // TODO: See above
+    //val threeSeventhsRateInst = Module(new RegisterModule)
+    // Fix peek-poke bridge under frequencies that aren't divisons of the base clock.
     //threeSeventhsRateInst.slowClock := threeSeventhsRate
-    threeSeventhsRateInst.slowClock := fullRate
-    threeSeventhsRateInst.in := halfRateInst.in
+    //threeSeventhsRateInst.slowClock := fullRate
+    //threeSeventhsRateInst.in := halfRateInst.in
 
     // TODO: Remove reset
     val peekPokeBridge = PeekPokeBridge(fullRate,
                                         reset,
                                         ("in", halfRateInst.in),
                                         ("halfOut",halfRateInst.out),
-                                        ("thirdOut", thirdRateInst.out),
-                                        ("threeSeventhsOut", threeSeventhsRateInst.out))
+                                        ("thirdOut", thirdRateInst.out))
+    //                                    ("threeSeventhsOut", threeSeventhsRateInst.out))
   }
 }