This does not mean that a release of 18.0 is imminent, only that the
main branch includes breaking changes.
Also updates the versioning scheme of the `@next` channel to include
the upcoming semver number, as well as the word "alpha" to indicate the
stability of the release.
- Before: 0.0.0-e0d9b28999
- After: 18.0.0-alpha-e0d9b28999
Uses the layout of the build artifact directory to infer the format
of a given file, and which lint rules to apply.
This has the effect of decoupling the lint build job from the existing
Rollup script, so that if we ever add additional post-processing, or
if we replace Rollup, it will still work.
But the immediate motivation is to replace the separate "stable" and
"experimental" lint-build jobs with a single combined job.
My personal workflow is to develop against the www-modern release
channel, with the variant flags enabled, because it encompasses the
largest set of features. Then I rely on CI to run the tests against
all the other configurations.
So in practice, I almost always run
```
yarn test -r=www-modern --variant TEST_FILE
```
instead of
```
yarn test TEST_FILE
```
So, I've updated the `yarn test` command to use those options
by default.
When running the publish workflow, either via the command line or
via the daily cron job, we should use a constant SHA instead of
whatever happens to be at the head of the main branch at the time the
workflow is run.
The difference is subtle: currently, the SHA is read at runtime,
each time the workflow is run. With this change, the SHA is read right
before the workflow is created and passed in as a constant parameter.
In practical terms, this means if a workflow is re-run via the CircleCI
web UI, it will always re-run using the same commit SHA as the original
workflow, instead of fetching the latest SHA from GitHub, which may
have changed.
Also avoids a race condition where the head SHA changes in between the
Next publish job and the Experimental publish job.
npm will sometimes fail if you try to concurrently publish two different
versions of the same package, even if they use different dist tags.
So instead of publishing to the Next and Experimental channels
simultaneously, we'll do them one after the other.
If we did want to speed up these publish workflows, we could paralellize
by package instead of by release channel.
We don't always keep the reconciler forks in sync (otherwise it we
wouldn't have forked it) but during periods when they are meant to be in
sync, we use this job to confirm there are no differences.
* Parallelize Flow in CI
We added more host configs recently, and we run all the checks in
sequence, so sometimes Flow ends up being the slowest CI job.
This splits the job across multiple processes.
* Fix environment variable typo
Co-authored-by: Ricky <rickhanlonii@gmail.com>
Co-authored-by: Ricky <rickhanlonii@gmail.com>
PR #20728 added a command to initiate a prerelease using CI, but it left
the publish job unimplemented. This fills in the publish job.
Uses an npm automation token for authorization, which bypasses the need
for a one-time password. The token is configured via CircleCI's
environment variable panel.
Currently, it will always publish the head of the main branch. If the
head has already been published, it will exit gracefully.
It does not yet support publishing arbitrary commits, though we could
easily add that. I don't know how important that use case is, because
for PR branches, you can use CodeSandbox CI instead. Or as a last
resort, run the publish script locally.
Always publishing from main is nice because it further incentivizes us
to keep main in a releasable state. It also takes the guesswork out of
publishing a prerelease that's in a broken state: as long as we don't
merge broken PRs, we're fine.
Adds a new CircleCI workflow, which I will use to publish prereleases
(`next` and `experimental`) for a given commit.
The CircleCI API doesn't yet support triggering a specific workflow, but
it does support triggering a pipeline. So as a workaround you can
triggger the entire pipeline and use parameters to disable everything
except the workflow you want. CircleCI recommends this workaround here:
https://support.circleci.com/hc/en-us/articles/360050351292-How-to-trigger-a-workflow-via-CircleCI-API-v2-
Eventually we can set this workflow to trigger on a cron schedule (once
per week, for example).
* build-combined: Fix bundle sizes path
* Output COMMIT_SHA in build directory
Alternative to parsing an arbitrary package's version number, or
its `build-info.json`.
* Remove CircleCI environment variable requirement
I think this only reason we needed this was to support passing any
job id to `--build`, instead of requiring the `process_artifacts` job.
And to do that you needed to use the workflows API endpoint, which
requires an API token.
But now that the `--commit` argument exists and automatically finds the
correct job id, we can just use that.
* Add CI job that gets base artifacts
Uses download-experimental script and places the base artifacts into
a top-level folder.
* Migrate sizebot to combined workflow
Replaces the two separate sizebot jobs (one for each channel, stable and
experimental) with a single combined job that outputs size information
for all bundles in a single GitHub comment.
I didn't attempt to simplify the output at all, but we should. I think
what I would do is remove our custom Rollup sizes plugin, and read the
sizes from the filesystem instead. We would lose some information about
the build configuration used to generate each artifact, but that can be
inferred from the filepath. For example, the filepath
`fb-www/ReactDOM-dev.classic.js` already tells us everything we need to
know about the artifact. Leaving this for a follow up.
* Move GitHub status check inside retry loop
The download script will poll the CircleCI endpoint until the build job
is complete; it should also poll the GitHub status endpoint if the
build job hasn't been spawned yet.
* Only run get_base_build on main branch
Fixes issue in the new build workflow where the experimental packages do
not include "experimental" in the version string. This was because the
previous approach relied on the RELEASE_CHANNEL environment variable,
which we are no longer setting in the outer CI job, since we use the
same job to build both channels. To solve, I moved the version
post-processing into the build script itself.
Only affects the new build workflow. Old workflow is unchanged.
Longer term, I would like to remove version numbers from the source
entirely, including the package.jsons. We should use a placeholder
instead; that's mostly how it already works, since the release script
swaps out the versions before we publish to stable.
The goal is to simplify our CI pipeline so that all configurations
are built and tested in a single workflow.
As a first step, this adds a new build script entry point that builds
both the experimental and stable release channels into a single
artifacts directory.
The script works by wrapping the existing build script (which only
builds a single release channel at a time), then post-processing the
results to match the desired filesystem layout. A future version of the
build script would output the files directly without post-processing.
Because many parts of our infra depend on the existing layout of the
build artifacts directory, I have left the old workflows untouched.
We can incremental migrate to the new layout, then delete the old
workflows after we've finished.
* Add branch to yarn cache key
* Add checksum check for workspace info
* Fix yaml
* Try moving the command
* How about here
* Just inline it
* i hate it here
* try reverting back
* Add run
* idk
* try inlining the command everywhere
* Create workspace_info.txt when we create the cache
* Delete the timestamp
This adds a new dimension similar to dom-relay. It's different from
"native" which would be Flight for RN without Relay.
This has some copy-pasta that's the same between the two Relay builds but
the key difference will be Metro and we're not quite sure what other
differences there will be yet.
* Deprecate old test script commands
* Update PR template test script
* Add test-stable and test-www-classic
* Update circle test names
* Rename test-www-classic to test-classic
* Missed some job renames
* Missed some more job renames
* Add new test cli
* Remove --variant accidentally added to test-persist
* s/test/tests
* Updates from review
* Update package.json tests
* Missed a release channel in circle.yaml
* Update config.yml to use just run: with test commands
* Update release-channel options and add build dir checks
* Update test args to use the new release-channel options
* Fix error in circle config.yml
* Fix a wrong condition for the --variant check
* Fix a wrong condition for the --persistent check
* Prettier
* Require build check for devtool tests as well
We need this so we can version them separately and use different
feature flags than we use for OSS RN.
I put them in a separate facebook-react-native folder which won't go
into the RN GH repo. I plan on moving the renderers there too but not yet.
* Lint bundles using the bundle config instead of scanning for files
This ensures that we look for all the files that we expect to see there.
If something doesn't get built we wouldn't detect it.
However, this doesn't find files that aren't part of our builds such as
indirection files in the root. This will need to change with ESM anyway
since indirection files doesn't work. Everything should be built anyway.
This ensures that we can use the bundles.js config to determine special
cases instead of relying on file system conventions.
* Run lint with flag
I made a mistake when setting these up a while ago. Setting the NODE_ENV
in the CircleCI config doesn't work because it's also set in the node
script command.
The number of test commands is getting out of control. Might need to fix
it at some point. Not today for me.
* Remove /dist/ UMD builds
We publish UMDs to npm (and we're considering stopping even that).
This means we'll stop publishing to http://react.zpao.com/builds/master/latest/
* Update fixture paths
In CI, we run our test suite against multiple build configurations. For
example, we run our tests in both dev and prod, and in both the
experimental and stable release channels. This is to prevent accidental
deviations in behavior between the different builds. If there's an
intentional deviation in behavior, the test author must account
for them.
However, we currently don't run tests against the www builds. That's
a problem, because it's common for features to land in www before they
land anywhere else, including the experimental release channel.
Typically we do this so we can gradually roll out the feature behind
a flag before deciding to enable it.
The way we test those features today is by mutating the
`shared/ReactFeatureFlags` module. There are a few downsides to this
approach, though. The flag is only overridden for the specific tests or
test suites where you apply the override. But usually what you want is
to run *all* tests with the flag enabled, to protect against unexpected
regressions.
Also, mutating the feature flags module only works when running the
tests against source, not against the final build artifacts, because the
ReactFeatureFlags module is inlined by the build script.
Instead, we should run the test suite against the www configuration,
just like we do for prod, experimental, and so on. I've added a new
command, `yarn test-www`. It automatically runs in CI.
Some of the www feature flags are dynamic; that is, they depend on
a runtime condition (i.e. a GK). These flags are imported from an
external module that lives in www. Those flags will be enabled for some
clients and disabled for others, so we should run the tests against
*both* modes.
So I've added a new global `__VARIANT__`, and a new test command `yarn
test-www-variant`. `__VARIANT__` is set to false by default; when
running `test-www-variant`, it's set to true.
If we were going for *really* comprehensive coverage, we would run the
tests against every possible configuration of feature flags: 2 ^
numberOfFlags total combinations. That's not practical, though, so
instead we only run against two combinations: once with `__VARIANT__`
set to `true`, and once with it set to `false`. We generally assume that
flags can be toggled independently, so in practice this should
be enough.
You can also refer to `__VARIANT__` in tests to detect which mode you're
running in. Or, you can import `shared/ReactFeatureFlags` and read the
specific flag you can about. However, we should stop mutating that
module going forward. Treat it as read-only.
In this commit, I have only setup the www tests to run against source.
I'll leave running against build for a follow up.
Many of our tests currently assume they run only in the default
configuration, and break when certain flags are toggled. Rather than fix
these all up front, I've hard-coded the relevant flags to the default
values. We can incrementally migrate those tests later.
* Skip abandoned project folders in Jest config
This fixes a problem that occurs after renaming a package.
* Fix test_build_devtools to run test-build-devtools
* Exclude console.error plugin for DevTools packages
* Use correct release channel for DevTools tests
This should fix the createRoot error.
* Fix TZ dependent test
* Change DT job dependencies
`yarn build` defaults to building in experimental mode. To opt-out, set
the `RELEASE_CHANNEL` environment variable to `stable`. This is the same
as what we do when running tests via `yarn test`, to make local
development easier.
* Tests run in experimental mode by default
For local development, you usually want experiments enabled. Unless
the release channel is set with an environment variable, tests will
run with __EXPERIMENTAL__ set to `true`.
* Remove concurrent APIs from stable builds
Those who want to try concurrent mode should use the experimental
builds instead.
I've left the `unstable_` prefixed APIs in the Facebook build so we
can continue experimenting with them internally without blessing them
for widespread use.
* Turn on SSR flags in experimental build
* Remove prefixed concurrent APIs from www build
Instead we'll use the experimental builds when syncing to www.
* Remove "canary" from internal React version string
* Don't bother including `unstable_` in error
The method names don't get stripped out of the production bundles
because they are passed as arguments to the error decoder.
Let's just always use the unprefixed APIs in the messages.
* Set up experimental builds
The experimental builds are packaged exactly like builds in the stable
release channel: same file structure, entry points, and npm package
names. The goal is to match what will eventually be released in stable
as closely as possible, but with additional features turned on.
Versioning and Releasing
------------------------
The experimental builds will be published to the same registry and
package names as the stable ones. However, they will be versioned using
a separate scheme. Instead of semver versions, experimental releases
will receive arbitrary version strings based on their content hashes.
The motivation is to thwart attempts to use a version range to match
against future experimental releases. The only way to install or depend
on an experimental release is to refer to the specific version number.
Building
--------
I did not use the existing feature flag infra to configure the
experimental builds. The reason is because feature flags are designed
to configure a single package. They're not designed to generate multiple
forks of the same package; for each set of feature flags, you must
create a separate package configuration.
Instead, I've added a new build dimension called the **release
channel**. By default, builds use the **stable** channel. There's
also an **experimental** release channel. We have the option to add more
in the future.
There are now two dimensions per artifact: build type (production,
development, or profiling), and release channel (stable or
experimental). These are separate dimensions because they are
combinatorial: there are stable and experimental production builds,
stable and experimental developmenet builds, and so on.
You can add something to an experimental build by gating on
`__EXPERIMENTAL__`, similar to how we use `__DEV__`. Anything inside
these branches will be excluded from the stable builds.
This gives us a low effort way to add experimental behavior in any
package without setting up feature flags or configuring a new package.
* warn when using the wrong renderer's act around another renderer's updates
like it says. it uses a real object as the sigil (instead of just a boolean). specifically, it uses a renderer's flushPassiveEffects as the sigil. We also run tests for this separate from our main suite (which doesn't allow loading multiple renderers in a suite), but makes sure to run this in CI as well.
* unneeded (and wrong) comment
* run the dom fixture on CI
* update the sigil only in __DEV__
* remove the obnoxious comment
* use an explicit export for the sigil
* Write size info to separate file per bundle
`bundle-sizes.json` contains the combined size information for every
build. This makes it easier to store and process, but it prevents us
from parallelizing the build script, because each process would need to
write to the same file.
So I've updated the Rollup script to output individual files per build.
A downstream CI job consolidates them into a single file.
I have not parallelized the Rollup script yet. I'll do that next.
* Parallelize the build script
Uses CircleCI's `parallelism` config option to spin up multiple build
processes.
Removes `--extract-errors` argument from CI build script command.
Instead, the author is expected to run `yarn extract-errors` locally
or manually edit the error code map.
The lint rule should be sufficient to catch unminified errors, but
as an extra precaution, I added a post-build step that greps the
production bundles. The post-build step works even if someone disables
the lint rule for a specific line or file.
The sizebot scrapes the GitHub `/statuses` endpoint to get the lastest
CircleCI build number for master, in order to fetch the bundle size
info for that build, which are stored as build artifacts. (There's gotta
be a better way to do this, but that's what we have for now.) This
updates the script to match the name of the updated CircleCI job that
generates the bundle sizes.
Updates the CircleCI config to use the workflows features to run jobs in
parallel, instead of the `parallelism` option. This change alone doesn't
improve the overall build time much, since almost all of the total time
is spent running the Rollup script, which runs entirely sequentially.
But it does improve reporting, and should make it easier to add
additional parallel jobs in the future.
* Store FB bundles as CI artifacts
Updates the Circle CI config to store Facebook bundles as build
artifacts. We already do this for our npm packages.
* Might as well store everything in build/
* Store build directory as a tarball
So it's easy to download
CI builds should always use the `--frozen-lockfile` option. It will fail the build if the lockfile is out-of-date:
> If you need reproducible dependencies, which is usually the case with the continuous integration systems, you should pass --frozen-lockfile flag.
(https://yarnpkg.com/en/docs/cli/install/)
* makes closure compiler threaded
* Dans PR with a closure compiler java version
* Remove unused dep
* Pin GCC
* Prettier
* Nit rename
* Fix error handling
* Name plugins consistently
* Fix lint
* Maybe this works?
* or this
* AppVeyor
* Fix lint
Andrew Clark