Compare commits

..

26 Commits

Author SHA1 Message Date
zeripath
bb054fdfa1 Changelog for 1.14.6 (#16619)
## [1.14.6](https://github.com/go-gitea/gitea/releases/tag/v1.14.6) - 2021-08-04

* SECURITY
  * Bump github.com/markbates/goth from v1.67.1 to v1.68.0 (#16538) (#16540)
  * Switch to maintained JWT lib (#16532) (#16535)
  * Upgrade to latest version of golang-jwt (as forked for 1.14) (#16590) (#16607)
* BUGFIXES
  * Add basic edit ldap auth test & actually fix #16252 (#16465) (#16495)
  * Make cancel from CatFileBatch and CatFileBatchCheck wait for the command to end (#16479) (#16481)

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-08-05 17:39:25 +01:00
zeripath
7760a7f385 Upgrade to latest version of golang-jwt (as forked for 1.14) (#16607)
* Forcibly update the vendored versions using a replace

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-08-04 00:21:00 +03:00
6543
3107c9dfc3 upgraded github.com/markbates/goth v1.67.1 => v1.68.0 (#16540) 2021-07-24 18:16:07 +01:00
6543
a66ff8a210 switch to maintained lib (#16535)
Co-authored-by: Matti R <matti@mdranta.net>
2021-07-24 11:13:56 -04:00
zeripath
6a3c7856c8 Add basic edit ldap auth test & actually fix #16252 (#16465) (#16495)
Backport #16465

One of the reasons why #16447 was needed and why #16268 was needed in
the first place was because it appears that editing ldap configuration
doesn't get tested.

This PR therefore adds a basic test that will run the edit pipeline.

In doing so it's now clear that #16447 and #16268 aren't actually
solving #16252. It turns out that what actually happens is that is that
the bytes are actually double encoded.

This PR now changes the json unmarshal wrapper to handle this double
encode.

Fix #16252

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: 6543 <6543@obermui.de>
2021-07-22 16:33:50 +03:00
zeripath
3299f044d3 Make cancel from CatFileBatch and CatFileBatchCheck wait for the command to end (#16479) (#16481)
* Make cancel from CatFileBatch and CatFileBatchCheck wait for the command to end

Fix #16427 (again!)

* handle sharing violation error code

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-07-21 18:34:53 +02:00
zeripath
e6c222511d Retry rename on lock induced failures (re-fix) (#16461) (#16463)
Backport #16461

Unfortunately #16435 asserts the wrong error and should use
os.LinkError not os.PathError.

Fix #16439

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-07-16 17:48:39 +01:00
zeripath
62fa153f9f Changelog for v1.14.5 (#16450)
Once #16449 is merged I think we should release 1.14.5. There are a couple of
security fixes and the broken #16268 is annoying enough that we should just release
things.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-07-16 07:32:00 +01:00
zeripath
be46f240d9 Fix crash following ldap authentication update (#16447) (#16449)
Backport #16447

Unfortunately #16268 contained a terrible error, whereby there was a double
indirection taken when unmarshalling the source data. This fatally breaks
authentication configuration reading.

Fix #16342

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-07-15 20:17:47 -04:00
zeripath
ca55e49cc0 Retry rename on lock induced failures (#16435) (#16439)
Backport #16435

Due to external locking on Windows it is possible for an
os.Rename to fail if the files or directories are being
used elsewhere.

This PR simply suggests retrying the rename again similar
to how we handle the os.Remove problems.

Fix #16427

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-07-15 22:57:51 +03:00
Norwin
58615be523 Validate issue index before querying DB (#16406) (#16410) 2021-07-12 16:22:47 -04:00
6543
6df82db0f7 Replace plugins/docker with techknowlogick/drone-dockerin ci (#16407) (#16409)
* plugins/docker -> techknowlogick/drone-docker

* It is multi-arch
2021-07-12 15:30:55 -04:00
6543
d98694e6ca Update bluemonday to v1.0.15 (#16379) (#16380)
* Update bluemonday to v1.0.15 (#16379)

* Fix TESTS
2021-07-09 02:47:27 +02:00
6543
ac0f452b30 Redirect on bad CSRF instead of presenting bad page (#14937) (#16378)
The current CSRF handler is a bit harsh with bad CSRF tokens on webpages
I think we can be a little kinder and redirect to base page with a flash error

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: zeripath <art27@cantab.net>
2021-07-08 20:12:20 +02:00
Kyle D
6e5fd5c584 Hide mirror passwords on repo settings page (#16022) (#16355) 2021-07-07 16:22:32 +02:00
Lunny Xiao
d0b8e3c8e1 Changelog for v1.14.4 (#16348) 2021-07-06 15:56:15 +03:00
6543
7ff8e863a5 Fix error message if user not exist (#16343)
Co-authored-by: Sergey Dryabzhinsky <sergey@rusoft.ru>
2021-07-05 23:58:47 +03:00
zeripath
c65e49d72f Fix relative links in postprocessed images (#16334) (#16340)
* Fix relative links in postprocessed images (#16334)

If a pre-post-processed file contains relative img tags these need to be updated
and joined correctly with the prefix. Finally, the node attributes need to be updated.

Fix #16308

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
2021-07-04 16:28:29 +02:00
6543
50084daa4c Fix list_options GetStartEnd (#16303) (#16305)
end is start + pageSize and not start + page

Co-authored-by: sebastian-sauer <sauer.sebastian@gmail.com>
2021-06-30 00:23:24 +01:00
6543
c7db7438b7 Fix API to return author for author on commits(#16276) (#16277) 2021-06-27 21:37:42 -04:00
zeripath
e11f042a95 Handle misencoding of login_source cfg in mssql (#16268) (#16275)
Backport #16268

Unfortunately due a bug in xorm (see https://gitea.com/xorm/xorm/pulls/1957) updating
loginsources on MSSQL causes them to become corrupted. (#16252)

Whilst waiting for the referenced PR to be merged and to handle the corrupted
loginsources correctly we need to add a wrapper to the `FromDB()` methods to look
for and ignore the misplaced BOMs that have been added.

Fix #16252

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-06-27 22:33:25 +01:00
KN4CK3R
87782636e6 Fixed issues not updated by commits (#16254) (#16261)
`UpdateIssuesCommit` may get called with fewer commits because of `FeedMaxCommitNum` and therefore may miss some commands.
2021-06-26 14:11:31 +02:00
zeripath
b935472cdf Improve efficiency in FindRenderizableReferenceNumeric and getReference (#16251) (#16255)
* Improve efficiency in FindRenderizableReferenceNumeric and getReferences (#16251)

* The Fuzzer is running on a non-repo urlprefix which is incorrect for RenderRaw
* Make FindRenderizableReferenceNumeric and getReferences more efficient

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>

* as per comment on original pr

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-06-26 08:31:03 +01:00
6543
8ac48584ec Use html.Parse rather than html.ParseFragment (#16223) (#16225)
* Use html.Parse rather than html.ParseFragment
  There have been a few issues with html.ParseFragment - just use html.Parse instead.

* Skip document node

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: zeripath <art27@cantab.net>
2021-06-22 09:46:39 +08:00
KN4CK3R
e898590c81 Update milestone counters on new issue (#16183) (#16224)
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: zeripath <art27@cantab.net>
2021-06-22 00:25:08 +02:00
zeripath
d407857d97 reqOrgMembership calls need to be preceded by reqToken (#16198) (#16219)
Backport #16198

ReqOrgMembership calls need to be preceded by reqToken

Fix #16192

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
2021-06-21 16:20:45 +01:00
238 changed files with 15496 additions and 12378 deletions

View File

@@ -709,7 +709,7 @@ steps:
- name: publish
pull: always
image: plugins/docker:linux-amd64
image: techknowlogick/drone-docker:latest
settings:
auto_tag: true
auto_tag_suffix: linux-amd64
@@ -726,7 +726,7 @@ steps:
- pull_request
- name: publish-rootless
image: plugins/docker:linux-amd64
image: techknowlogick/drone-docker:latest
settings:
dockerfile: Dockerfile.rootless
auto_tag: true
@@ -764,7 +764,7 @@ trigger:
steps:
- name: dryrun
pull: always
image: plugins/docker:linux-arm64
image: techknowlogick/drone-docker:latest
settings:
dry_run: true
repo: gitea/gitea
@@ -806,7 +806,7 @@ steps:
- name: publish
pull: always
image: plugins/docker:linux-arm64
image: techknowlogick/drone-docker:latest
settings:
auto_tag: true
auto_tag_suffix: linux-arm64
@@ -826,7 +826,7 @@ steps:
- pull_request
- name: publish-rootless
image: plugins/docker:linux-arm64
image: techknowlogick/drone-docker:latest
settings:
dockerfile: Dockerfile.rootless
auto_tag: true

View File

@@ -4,6 +4,41 @@ This changelog goes through all the changes that have been made in each release
without substantial changes to our git log; to see the highlights of what has
been added to each release, please refer to the [blog](https://blog.gitea.io).
## [1.14.6](https://github.com/go-gitea/gitea/releases/tag/v1.14.6) - 2021-08-04
* SECURITY
* Bump github.com/markbates/goth from v1.67.1 to v1.68.0 (#16538) (#16540)
* Switch to maintained JWT lib (#16532) (#16535)
* Upgrade to latest version of golang-jwt (as forked for 1.14) (#16590) (#16607)
* BUGFIXES
* Add basic edit ldap auth test & actually fix #16252 (#16465) (#16495)
* Make cancel from CatFileBatch and CatFileBatchCheck wait for the command to end (#16479) (#16481)
## [1.14.5](https://github.com/go-gitea/gitea/releases/tag/v1.14.5) - 2021-07-16
* SECURITY
* Hide mirror passwords on repo settings page (#16022) (#16355)
* Update bluemonday to v1.0.15 (#16379) (#16380)
* BUGFIXES
* Retry rename on lock induced failures (#16435) (#16439)
* Validate issue index before querying DB (#16406) (#16410)
* Fix crash following ldap authentication update (#16447) (#16449)
* ENHANCEMENTS
* Redirect on bad CSRF instead of presenting bad page (#14937) (#16378)
## [1.14.4](https://github.com/go-gitea/gitea/releases/tag/v1.14.4) - 2021-07-06
* BUGFIXES
* Fix relative links in postprocessed images (#16334) (#16340)
* Fix list_options GetStartEnd (#16303) (#16305)
* Fix API to use author for commits instead of committer (#16276) (#16277)
* Handle misencoding of login_source cfg in mssql (#16268) (#16275)
* Fixed issues not updated by commits (#16254) (#16261)
* Improve efficiency in FindRenderizableReferenceNumeric and getReference (#16251) (#16255)
* Use html.Parse rather than html.ParseFragment (#16223) (#16225)
* Fix milestone counters on new issue (#16183) (#16224)
* reqOrgMembership calls need to be preceded by reqToken (#16198) (#16219)
## [1.14.3](https://github.com/go-gitea/gitea/releases/tag/v1.14.3) - 2021-06-10
* SECURITY

View File

@@ -19,6 +19,7 @@ import (
"code.gitea.io/gitea/modules/public"
"code.gitea.io/gitea/modules/setting"
"code.gitea.io/gitea/modules/templates"
"code.gitea.io/gitea/modules/util"
"github.com/gobwas/glob"
"github.com/urfave/cli"
@@ -271,7 +272,7 @@ func extractAsset(d string, a asset, overwrite, rename bool) error {
} else if !fi.Mode().IsRegular() {
return fmt.Errorf("%s already exists, but it's not a regular file", dest)
} else if rename {
if err := os.Rename(dest, dest+".bak"); err != nil {
if err := util.Rename(dest, dest+".bak"); err != nil {
return fmt.Errorf("Error creating backup for %s: %v", dest, err)
}
// Attempt to respect file permissions mask (even if user:group will be set anew)

View File

@@ -23,7 +23,7 @@ import (
"code.gitea.io/gitea/modules/private"
"code.gitea.io/gitea/modules/setting"
"github.com/dgrijalva/jwt-go"
"github.com/golang-jwt/jwt"
jsoniter "github.com/json-iterator/go"
"github.com/kballard/go-shellquote"
"github.com/urfave/cli"

14
go.mod
View File

@@ -28,7 +28,6 @@ require (
github.com/couchbase/goutils v0.0.0-20210118111533-e33d3ffb5401 // indirect
github.com/cpuguy83/go-md2man/v2 v2.0.0 // indirect
github.com/denisenkom/go-mssqldb v0.9.0
github.com/dgrijalva/jwt-go v3.2.0+incompatible
github.com/dlclark/regexp2 v1.4.0 // indirect
github.com/dustin/go-humanize v1.0.0
github.com/editorconfig/editorconfig-core-go/v2 v2.4.1
@@ -53,6 +52,7 @@ require (
github.com/gogs/chardet v0.0.0-20191104214054-4b6791f73a28
github.com/gogs/cron v0.0.0-20171120032916-9f6c956d3e14
github.com/gogs/go-gogs-client v0.0.0-20210131175652-1d7215cd8d85
github.com/golang-jwt/jwt v3.2.1+incompatible
github.com/golang/snappy v0.0.3 // indirect
github.com/google/go-github/v32 v32.1.0
github.com/google/uuid v1.2.0
@@ -78,7 +78,7 @@ require (
github.com/libdns/libdns v0.2.0 // indirect
github.com/lunny/dingtalk_webhook v0.0.0-20171025031554-e3534c89ef96
github.com/mailru/easyjson v0.7.7 // indirect
github.com/markbates/goth v1.67.1
github.com/markbates/goth v1.68.0
github.com/mattn/go-isatty v0.0.12
github.com/mattn/go-runewidth v0.0.10 // indirect
github.com/mattn/go-sqlite3 v1.14.6
@@ -86,7 +86,7 @@ require (
github.com/mgechev/revive v1.0.3
github.com/mholt/acmez v0.1.3 // indirect
github.com/mholt/archiver/v3 v3.5.0
github.com/microcosm-cc/bluemonday v1.0.7
github.com/microcosm-cc/bluemonday v1.0.15
github.com/miekg/dns v1.1.40 // indirect
github.com/minio/md5-simd v1.1.2 // indirect
github.com/minio/minio-go/v7 v7.0.10
@@ -136,10 +136,10 @@ require (
go.uber.org/multierr v1.6.0 // indirect
go.uber.org/zap v1.16.0 // indirect
golang.org/x/crypto v0.0.0-20210220033148-5ea612d1eb83
golang.org/x/net v0.0.0-20210405180319-a5a99cb37ef4
golang.org/x/net v0.0.0-20210614182718-04defd469f4e
golang.org/x/oauth2 v0.0.0-20210220000619-9bb904979d93
golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44
golang.org/x/text v0.3.5
golang.org/x/sys v0.0.0-20210423082822-04245dca01da
golang.org/x/text v0.3.6
golang.org/x/time v0.0.0-20210220033141-f8bda1e9f3ba // indirect
golang.org/x/tools v0.1.0
gopkg.in/alexcesaro/quotedprintable.v3 v3.0.0-20150716171945-2caba252f4dc // indirect
@@ -153,3 +153,5 @@ require (
)
replace github.com/hashicorp/go-version => github.com/6543/go-version v1.2.4
replace github.com/golang-jwt/jwt v3.2.1+incompatible => github.com/zeripath/jwt v3.2.2-go1.14+incompatible

26
go.sum
View File

@@ -127,8 +127,9 @@ github.com/asaskevich/govalidator v0.0.0-20180720115003-f9ffefc3facf/go.mod h1:l
github.com/asaskevich/govalidator v0.0.0-20190424111038-f61b66f89f4a/go.mod h1:lB+ZfQJz7igIIfQNfa7Ml4HSf2uFQQRzpGGRXenZAgY=
github.com/asaskevich/govalidator v0.0.0-20200108200545-475eaeb16496/go.mod h1:oGkLhpf+kjZl6xBf758TQhh5XrAeiJv/7FRz/2spLIg=
github.com/asaskevich/govalidator v0.0.0-20200428143746-21a406dcc535/go.mod h1:oGkLhpf+kjZl6xBf758TQhh5XrAeiJv/7FRz/2spLIg=
github.com/asaskevich/govalidator v0.0.0-20200907205600-7a23bdc65eef h1:46PFijGLmAjMPwCCCo7Jf0W6f9slllCkkv7vyc1yOSg=
github.com/asaskevich/govalidator v0.0.0-20200907205600-7a23bdc65eef/go.mod h1:WaHUgvxTVq04UNunO+XhnAqY/wQc+bxr74GqbsZ/Jqw=
github.com/asaskevich/govalidator v0.0.0-20210307081110-f21760c49a8d h1:Byv0BzEl3/e6D5CLfI0j/7hiIEtvGVFPCZ7Ei2oq8iQ=
github.com/asaskevich/govalidator v0.0.0-20210307081110-f21760c49a8d/go.mod h1:WaHUgvxTVq04UNunO+XhnAqY/wQc+bxr74GqbsZ/Jqw=
github.com/aws/aws-lambda-go v1.13.3/go.mod h1:4UKl9IzQMoD+QF79YdCuzCwp8VbmG4VAQwij/eHl5CU=
github.com/aws/aws-sdk-go v1.27.0/go.mod h1:KmX6BPdI08NWTb3/sm4ZGu5ShLoqVDhKgpiN924inxo=
github.com/aws/aws-sdk-go v1.34.28/go.mod h1:H7NKnBqNVzoTJpGfLrQkkD+ytBA93eiDYi/+8rV9s48=
@@ -250,7 +251,6 @@ github.com/denisenkom/go-mssqldb v0.0.0-20191128021309-1d7a30a10f73/go.mod h1:xb
github.com/denisenkom/go-mssqldb v0.0.0-20200428022330-06a60b6afbbc/go.mod h1:xbL0rPBG9cCiLr28tMa8zpbdarY27NDyej4t/EjAShU=
github.com/denisenkom/go-mssqldb v0.9.0 h1:RSohk2RsiZqLZ0zCjtfn3S4Gp4exhpBWHyQ7D0yGjAk=
github.com/denisenkom/go-mssqldb v0.9.0/go.mod h1:xbL0rPBG9cCiLr28tMa8zpbdarY27NDyej4t/EjAShU=
github.com/dgrijalva/jwt-go v3.2.0+incompatible h1:7qlOGliEKZXTDg6OTjfoBKDXWrumCAMpl/TFQ4/5kLM=
github.com/dgrijalva/jwt-go v3.2.0+incompatible/go.mod h1:E3ru+11k8xSBh+hMPgOLZmtrrCbhqsmaPHjLKYnJCaQ=
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f h1:lO4WD4F/rVNCu3HqELle0jiPLLBs70cWOduZpkS1E78=
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f/go.mod h1:cuUVRXasLTGF7a8hSLbxyZXjz+1KgoB3wDUb6vlszIc=
@@ -792,8 +792,8 @@ github.com/mailru/easyjson v0.7.6/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJ
github.com/mailru/easyjson v0.7.7 h1:UGYAvKxe3sBsEDzO8ZeWOSlIQfWFlxbzLZe7hwFURr0=
github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc=
github.com/markbates/going v1.0.0/go.mod h1:I6mnB4BPnEeqo85ynXIx1ZFLLbtiLHNXVgWeFO9OGOA=
github.com/markbates/goth v1.67.1 h1:gU5B0pzHVyhnJPwGynfFnkfvaQ39C1Sy+ewdl+bhAOw=
github.com/markbates/goth v1.67.1/go.mod h1:EyLFHGU5ySr2GXRDyJH5nu2dA7parbC8QwIYW/rGcWg=
github.com/markbates/goth v1.68.0 h1:90sKvjRAKHcl9V2uC9x/PJXeD78cFPiBsyP1xVhoQfA=
github.com/markbates/goth v1.68.0/go.mod h1:V2VcDMzDiMHW+YmqYl7i0cMiAUeCkAe4QE6jRKBhXZw=
github.com/markbates/oncer v0.0.0-20181203154359-bf2de49a0be2/go.mod h1:Ld9puTsIW75CHf65OeIOkyKbteujpZVXDpWK6YGZbxE=
github.com/markbates/safe v1.0.1/go.mod h1:nAqgmRi7cY2nqMc92/bSEeQA+R4OheNU2T1kNSCBdG0=
github.com/mattn/go-colorable v0.0.9/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaOChaDxuIBZU=
@@ -830,8 +830,8 @@ github.com/mholt/acmez v0.1.3 h1:J7MmNIk4Qf9b8mAGqAh4XkNeowv3f1zW816yf4zt7Qk=
github.com/mholt/acmez v0.1.3/go.mod h1:8qnn8QA/Ewx8E3ZSsmscqsIjhhpxuy9vqdgbX2ceceM=
github.com/mholt/archiver/v3 v3.5.0 h1:nE8gZIrw66cu4osS/U7UW7YDuGMHssxKutU8IfWxwWE=
github.com/mholt/archiver/v3 v3.5.0/go.mod h1:qqTTPUK/HZPFgFQ/TJ3BzvTpF/dPtFVJXdQbCmeMxwc=
github.com/microcosm-cc/bluemonday v1.0.7 h1:6yAQfk4XT+PI/dk1ZeBp1gr3Q2Hd1DR0O3aEyPUJVTE=
github.com/microcosm-cc/bluemonday v1.0.7/go.mod h1:HOT/6NaBlR0f9XlxD3zolN6Z3N8Lp4pvhp+jLS5ihnI=
github.com/microcosm-cc/bluemonday v1.0.15 h1:J4uN+qPng9rvkBZBoBb8YGR+ijuklIMpSOZZLjYpbeY=
github.com/microcosm-cc/bluemonday v1.0.15/go.mod h1:ZLvAzeakRwrGnzQEvstVzVt3ZpqOF2+sdFr0Om+ce30=
github.com/miekg/dns v1.0.14/go.mod h1:W1PPwlIAgtquWBMBEV9nkV9Cazfe8ScdGz/Lj7v3Nrg=
github.com/miekg/dns v1.1.30/go.mod h1:KNUDUusw/aVsxyTYZM1oqvCicbwhgbNgztCETuNZ7xM=
github.com/miekg/dns v1.1.40 h1:pyyPFfGMnciYUk/mXpKkVmeMQjfXqt3FAJ2hy7tPiLA=
@@ -1152,6 +1152,8 @@ github.com/yuin/goldmark-highlighting v0.0.0-20200307114337-60d527fdb691/go.mod
github.com/yuin/goldmark-meta v1.0.0 h1:ScsatUIT2gFS6azqzLGUjgOnELsBOxMXerM3ogdJhAM=
github.com/yuin/goldmark-meta v1.0.0/go.mod h1:zsNNOrZ4nLuyHAJeLQEZcQat8dm70SmB2kHbls092Gc=
github.com/zenazn/goji v0.9.0/go.mod h1:7S9M489iMyHBNxwZnk9/EHS098H4/F6TATF2mIxtB1Q=
github.com/zeripath/jwt v3.2.2-go1.14+incompatible h1:jqxA3KuCQRLn0lHdt1G8t1EUJ92FmRUFnXHghVvJLJs=
github.com/zeripath/jwt v3.2.2-go1.14+incompatible/go.mod h1:pYPrRXN84mQC6u5c/08icdKllASIBEOurvsTPbDurLs=
github.com/ziutek/mymysql v1.5.4/go.mod h1:LMSpPZ6DbqWFxNCHW77HeMg9I646SAhApZ/wKdgO/C0=
go.etcd.io/bbolt v1.3.2/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU=
go.etcd.io/bbolt v1.3.3/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU=
@@ -1321,9 +1323,8 @@ golang.org/x/net v0.0.0-20201202161906-c7110b5ffcbb/go.mod h1:sp8m0HH+o8qH0wwXwY
golang.org/x/net v0.0.0-20201209123823-ac852fbbde11/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20201224014010-6772e930b67b/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20210119194325-5f4716e94777/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20210331212208-0fccb6fa2b5c/go.mod h1:p54w0d4576C0XHj96bSt6lcn1PtDYWL6XObtHCRCNQM=
golang.org/x/net v0.0.0-20210405180319-a5a99cb37ef4 h1:4nGaVu0QrbjT/AK2PRLuQfQuh6DJve+pELhqTdAj3x0=
golang.org/x/net v0.0.0-20210405180319-a5a99cb37ef4/go.mod h1:p54w0d4576C0XHj96bSt6lcn1PtDYWL6XObtHCRCNQM=
golang.org/x/net v0.0.0-20210614182718-04defd469f4e h1:XpT3nA5TvE525Ne3hInMh6+GETgn27Zfm9dxsThnX2Q=
golang.org/x/net v0.0.0-20210614182718-04defd469f4e/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20181106182150-f42d05182288/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
@@ -1419,8 +1420,8 @@ golang.org/x/sys v0.0.0-20210112080510-489259a85091/go.mod h1:h1NjWce9XRLGQEsW7w
golang.org/x/sys v0.0.0-20210119212857-b64e53b001e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210220050731-9a76102bfb43/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44 h1:Bli41pIlzTzf3KEY06n+xnzK/BESIg2ze4Pgfh/aI8c=
golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210423082822-04245dca01da h1:b3NXsE2LusjYGGjL5bxEVZZORm/YEFFrWFjR8eFrw/c=
golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1 h1:v+OssWQX+hTHEmOBgwxdZxK4zHq3yOs8F9J7mk0PY8E=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
@@ -1430,8 +1431,9 @@ golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.5 h1:i6eZZ+zk0SOf0xgBpEpPD18qWcJda6q1sxt3S0kzyUQ=
golang.org/x/text v0.3.5/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.6 h1:aRYxNxv6iGQlyVaZmk6ZgYEDa+Jg18DxebPSrd6bg1M=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/time v0.0.0-20180412165947-fbb02b2291d2/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=

View File

@@ -144,7 +144,9 @@ func TestAPITeamSearch(t *testing.T) {
var results TeamSearchResults
session := loginUser(t, user.Name)
csrf := GetCSRF(t, session, "/"+org.Name)
req := NewRequestf(t, "GET", "/api/v1/orgs/%s/teams/search?q=%s", org.Name, "_team")
req.Header.Add("X-Csrf-Token", csrf)
resp := session.MakeRequest(t, req, http.StatusOK)
DecodeJSON(t, resp, &results)
assert.NotEmpty(t, results.Data)
@@ -154,7 +156,9 @@ func TestAPITeamSearch(t *testing.T) {
// no access if not organization member
user5 := models.AssertExistsAndLoadBean(t, &models.User{ID: 5}).(*models.User)
session = loginUser(t, user5.Name)
csrf = GetCSRF(t, session, "/"+org.Name)
req = NewRequestf(t, "GET", "/api/v1/orgs/%s/teams/search?q=%s", org.Name, "team")
req.Header.Add("X-Csrf-Token", csrf)
resp = session.MakeRequest(t, req, http.StatusForbidden)
}

View File

@@ -144,6 +144,60 @@ func TestLDAPUserSignin(t *testing.T) {
assert.Equal(t, u.Email, htmlDoc.Find(`label[for="email"]`).Siblings().First().Text())
}
func TestLDAPAuthChange(t *testing.T) {
defer prepareTestEnv(t)()
addAuthSourceLDAP(t, "")
session := loginUser(t, "user1")
req := NewRequest(t, "GET", "/admin/auths")
resp := session.MakeRequest(t, req, http.StatusOK)
doc := NewHTMLParser(t, resp.Body)
href, exists := doc.Find("table.table td a").Attr("href")
if !exists {
assert.True(t, exists, "No authentication source found")
return
}
req = NewRequest(t, "GET", href)
resp = session.MakeRequest(t, req, http.StatusOK)
doc = NewHTMLParser(t, resp.Body)
csrf := doc.GetCSRF()
host, _ := doc.Find(`input[name="host"]`).Attr("value")
assert.Equal(t, host, getLDAPServerHost())
binddn, _ := doc.Find(`input[name="bind_dn"]`).Attr("value")
assert.Equal(t, binddn, "uid=gitea,ou=service,dc=planetexpress,dc=com")
req = NewRequestWithValues(t, "POST", href, map[string]string{
"_csrf": csrf,
"type": "2",
"name": "ldap",
"host": getLDAPServerHost(),
"port": "389",
"bind_dn": "uid=gitea,ou=service,dc=planetexpress,dc=com",
"bind_password": "password",
"user_base": "ou=people,dc=planetexpress,dc=com",
"filter": "(&(objectClass=inetOrgPerson)(memberOf=cn=git,ou=people,dc=planetexpress,dc=com)(uid=%s))",
"admin_filter": "(memberOf=cn=admin_staff,ou=people,dc=planetexpress,dc=com)",
"restricted_filter": "(uid=leela)",
"attribute_username": "uid",
"attribute_name": "givenName",
"attribute_surname": "sn",
"attribute_mail": "mail",
"attribute_ssh_public_key": "",
"is_sync_enabled": "on",
"is_active": "on",
})
session.MakeRequest(t, req, http.StatusFound)
req = NewRequest(t, "GET", href)
resp = session.MakeRequest(t, req, http.StatusOK)
doc = NewHTMLParser(t, resp.Body)
host, _ = doc.Find(`input[name="host"]`).Attr("value")
assert.Equal(t, host, getLDAPServerHost())
binddn, _ = doc.Find(`input[name="bind_dn"]`).Attr("value")
assert.Equal(t, binddn, "uid=gitea,ou=service,dc=planetexpress,dc=com")
}
func TestLDAPUserSync(t *testing.T) {
if skipLDAPTests() {
t.Skip()

View File

@@ -11,6 +11,7 @@ import (
"strings"
"testing"
"code.gitea.io/gitea/modules/setting"
"code.gitea.io/gitea/modules/test"
"github.com/stretchr/testify/assert"
@@ -134,5 +135,13 @@ func TestCreateBranchInvalidCSRF(t *testing.T) {
"_csrf": "fake_csrf",
"new_branch_name": "test",
})
session.MakeRequest(t, req, http.StatusBadRequest)
resp := session.MakeRequest(t, req, http.StatusFound)
loc := resp.Header().Get("Location")
assert.Equal(t, setting.AppSubURL+"/", loc)
resp = session.MakeRequest(t, NewRequest(t, "GET", loc), http.StatusOK)
htmlDoc := NewHTMLParser(t, resp.Body)
assert.Equal(t,
"Bad Request: Invalid CSRF token",
strings.TrimSpace(htmlDoc.doc.Find(".ui.message").Text()),
)
}

View File

@@ -141,6 +141,12 @@ func (milestone *Milestone) checkForConsistency(t *testing.T) {
actual := getCount(t, x.Where("is_closed=?", true), &Issue{MilestoneID: milestone.ID})
assert.EqualValues(t, milestone.NumClosedIssues, actual,
"Unexpected number of closed issues for milestone %+v", milestone)
completeness := 0
if milestone.NumIssues > 0 {
completeness = milestone.NumClosedIssues * 100 / milestone.NumIssues
}
assert.Equal(t, completeness, milestone.Completeness)
}
func (label *Label) checkForConsistency(t *testing.T) {

View File

@@ -648,8 +648,10 @@ func (issue *Issue) doChangeStatus(e *xorm.Session, doer *User, isMergePull bool
}
// Update issue count of milestone
if err := updateMilestoneClosedNum(e, issue.MilestoneID); err != nil {
return nil, err
if issue.MilestoneID > 0 {
if err := updateMilestoneCounters(e, issue.MilestoneID); err != nil {
return nil, err
}
}
if err := issue.updateClosedNum(e); err != nil {
@@ -912,7 +914,7 @@ func newIssue(e *xorm.Session, doer *User, opts NewIssueOptions) (err error) {
opts.Issue.Index = inserted.Index
if opts.Issue.MilestoneID > 0 {
if _, err = e.Exec("UPDATE `milestone` SET num_issues=num_issues+1 WHERE id=?", opts.Issue.MilestoneID); err != nil {
if err := updateMilestoneCounters(e, opts.Issue.MilestoneID); err != nil {
return err
}
@@ -1032,6 +1034,9 @@ func newIssueAttempt(repo *Repository, issue *Issue, labelIDs []int64, uuids []s
// GetIssueByIndex returns raw issue without loading attributes by index in a repository.
func GetIssueByIndex(repoID, index int64) (*Issue, error) {
if index < 1 {
return nil, ErrIssueNotExist{}
}
issue := &Issue{
RepoID: repoID,
Index: index,

View File

@@ -129,8 +129,12 @@ func GetMilestoneByRepoIDANDName(repoID int64, name string) (*Milestone, error)
// GetMilestoneByID returns the milestone via id .
func GetMilestoneByID(id int64) (*Milestone, error) {
return getMilestoneByID(x, id)
}
func getMilestoneByID(e Engine, id int64) (*Milestone, error) {
var m Milestone
has, err := x.ID(id).Get(&m)
has, err := e.ID(id).Get(&m)
if err != nil {
return nil, err
} else if !has {
@@ -155,10 +159,6 @@ func UpdateMilestone(m *Milestone, oldIsClosed bool) error {
return err
}
if err := updateMilestoneCompleteness(sess, m.ID); err != nil {
return err
}
// if IsClosed changed, update milestone numbers of repository
if oldIsClosed != m.IsClosed {
if err := updateRepoMilestoneNum(sess, m.RepoID); err != nil {
@@ -171,23 +171,31 @@ func UpdateMilestone(m *Milestone, oldIsClosed bool) error {
func updateMilestone(e Engine, m *Milestone) error {
m.Name = strings.TrimSpace(m.Name)
_, err := e.ID(m.ID).AllCols().
_, err := e.ID(m.ID).AllCols().Update(m)
if err != nil {
return err
}
return updateMilestoneCounters(e, m.ID)
}
// updateMilestoneCounters calculates NumIssues, NumClosesIssues and Completeness
func updateMilestoneCounters(e Engine, id int64) error {
_, err := e.ID(id).
SetExpr("num_issues", builder.Select("count(*)").From("issue").Where(
builder.Eq{"milestone_id": m.ID},
builder.Eq{"milestone_id": id},
)).
SetExpr("num_closed_issues", builder.Select("count(*)").From("issue").Where(
builder.Eq{
"milestone_id": m.ID,
"milestone_id": id,
"is_closed": true,
},
)).
Update(m)
return err
}
func updateMilestoneCompleteness(e Engine, milestoneID int64) error {
_, err := e.Exec("UPDATE `milestone` SET completeness=100*num_closed_issues/(CASE WHEN num_issues > 0 THEN num_issues ELSE 1 END) WHERE id=?",
milestoneID,
Update(&Milestone{})
if err != nil {
return err
}
_, err = e.Exec("UPDATE `milestone` SET completeness=100*num_closed_issues/(CASE WHEN num_issues > 0 THEN num_issues ELSE 1 END) WHERE id=?",
id,
)
return err
}
@@ -256,25 +264,15 @@ func changeMilestoneAssign(e *xorm.Session, doer *User, issue *Issue, oldMilesto
}
if oldMilestoneID > 0 {
if err := updateMilestoneTotalNum(e, oldMilestoneID); err != nil {
if err := updateMilestoneCounters(e, oldMilestoneID); err != nil {
return err
}
if issue.IsClosed {
if err := updateMilestoneClosedNum(e, oldMilestoneID); err != nil {
return err
}
}
}
if issue.MilestoneID > 0 {
if err := updateMilestoneTotalNum(e, issue.MilestoneID); err != nil {
if err := updateMilestoneCounters(e, issue.MilestoneID); err != nil {
return err
}
if issue.IsClosed {
if err := updateMilestoneClosedNum(e, issue.MilestoneID); err != nil {
return err
}
}
}
if oldMilestoneID > 0 || issue.MilestoneID > 0 {
@@ -558,29 +556,6 @@ func updateRepoMilestoneNum(e Engine, repoID int64) error {
return err
}
func updateMilestoneTotalNum(e Engine, milestoneID int64) (err error) {
if _, err = e.Exec("UPDATE `milestone` SET num_issues=(SELECT count(*) FROM issue WHERE milestone_id=?) WHERE id=?",
milestoneID,
milestoneID,
); err != nil {
return
}
return updateMilestoneCompleteness(e, milestoneID)
}
func updateMilestoneClosedNum(e Engine, milestoneID int64) (err error) {
if _, err = e.Exec("UPDATE `milestone` SET num_closed_issues=(SELECT count(*) FROM issue WHERE milestone_id=? AND is_closed=?) WHERE id=?",
milestoneID,
true,
milestoneID,
); err != nil {
return
}
return updateMilestoneCompleteness(e, milestoneID)
}
// _____ _ _ _____ _
// |_ _| __ __ _ ___| | _____ __| |_ _(_)_ __ ___ ___ ___
// | || '__/ _` |/ __| |/ / _ \/ _` | | | | | '_ ` _ \ / _ \/ __|

View File

@@ -215,7 +215,7 @@ func TestChangeMilestoneStatus(t *testing.T) {
CheckConsistencyFor(t, &Repository{ID: milestone.RepoID}, &Milestone{})
}
func TestUpdateMilestoneClosedNum(t *testing.T) {
func TestUpdateMilestoneCounters(t *testing.T) {
assert.NoError(t, PrepareTestDatabase())
issue := AssertExistsAndLoadBean(t, &Issue{MilestoneID: 1},
"is_closed=0").(*Issue)
@@ -224,14 +224,14 @@ func TestUpdateMilestoneClosedNum(t *testing.T) {
issue.ClosedUnix = timeutil.TimeStampNow()
_, err := x.ID(issue.ID).Cols("is_closed", "closed_unix").Update(issue)
assert.NoError(t, err)
assert.NoError(t, updateMilestoneClosedNum(x, issue.MilestoneID))
assert.NoError(t, updateMilestoneCounters(x, issue.MilestoneID))
CheckConsistencyFor(t, &Milestone{})
issue.IsClosed = false
issue.ClosedUnix = 0
_, err = x.ID(issue.ID).Cols("is_closed", "closed_unix").Update(issue)
assert.NoError(t, err)
assert.NoError(t, updateMilestoneClosedNum(x, issue.MilestoneID))
assert.NoError(t, updateMilestoneCounters(x, issue.MilestoneID))
CheckConsistencyFor(t, &Milestone{})
}

View File

@@ -41,7 +41,7 @@ func (opts *ListOptions) setEnginePagination(e Engine) Engine {
func (opts *ListOptions) GetStartEnd() (start, end int) {
opts.setDefaultValues()
start = (opts.Page - 1) * opts.PageSize
end = start + opts.Page
end = start + opts.PageSize
return
}

View File

@@ -7,6 +7,7 @@ package models
import (
"crypto/tls"
"encoding/binary"
"errors"
"fmt"
"net/smtp"
@@ -69,6 +70,36 @@ var (
_ convert.Conversion = &SSPIConfig{}
)
// jsonUnmarshalHandleDoubleEncode - due to a bug in xorm (see https://gitea.com/xorm/xorm/pulls/1957) - it's
// possible that a Blob may be double encoded or gain an unwanted prefix of 0xff 0xfe.
func jsonUnmarshalHandleDoubleEncode(bs []byte, v interface{}) error {
json := jsoniter.ConfigCompatibleWithStandardLibrary
err := json.Unmarshal(bs, v)
if err != nil {
ok := true
rs := []byte{}
temp := make([]byte, 2)
for _, rn := range string(bs) {
if rn > 0xffff {
ok = false
break
}
binary.LittleEndian.PutUint16(temp, uint16(rn))
rs = append(rs, temp...)
}
if ok {
if rs[0] == 0xff && rs[1] == 0xfe {
rs = rs[2:]
}
err = json.Unmarshal(rs, v)
}
}
if err != nil && len(bs) > 2 && bs[0] == 0xff && bs[1] == 0xfe {
err = json.Unmarshal(bs[2:], v)
}
return err
}
// LDAPConfig holds configuration for LDAP login source.
type LDAPConfig struct {
*ldap.Source
@@ -76,8 +107,7 @@ type LDAPConfig struct {
// FromDB fills up a LDAPConfig from serialized format.
func (cfg *LDAPConfig) FromDB(bs []byte) error {
json := jsoniter.ConfigCompatibleWithStandardLibrary
return json.Unmarshal(bs, &cfg)
return jsonUnmarshalHandleDoubleEncode(bs, &cfg)
}
// ToDB exports a LDAPConfig to a serialized format.
@@ -104,8 +134,7 @@ type SMTPConfig struct {
// FromDB fills up an SMTPConfig from serialized format.
func (cfg *SMTPConfig) FromDB(bs []byte) error {
json := jsoniter.ConfigCompatibleWithStandardLibrary
return json.Unmarshal(bs, cfg)
return jsonUnmarshalHandleDoubleEncode(bs, cfg)
}
// ToDB exports an SMTPConfig to a serialized format.
@@ -122,8 +151,7 @@ type PAMConfig struct {
// FromDB fills up a PAMConfig from serialized format.
func (cfg *PAMConfig) FromDB(bs []byte) error {
json := jsoniter.ConfigCompatibleWithStandardLibrary
return json.Unmarshal(bs, &cfg)
return jsonUnmarshalHandleDoubleEncode(bs, cfg)
}
// ToDB exports a PAMConfig to a serialized format.
@@ -144,8 +172,7 @@ type OAuth2Config struct {
// FromDB fills up an OAuth2Config from serialized format.
func (cfg *OAuth2Config) FromDB(bs []byte) error {
json := jsoniter.ConfigCompatibleWithStandardLibrary
return json.Unmarshal(bs, cfg)
return jsonUnmarshalHandleDoubleEncode(bs, cfg)
}
// ToDB exports an SMTPConfig to a serialized format.
@@ -165,8 +192,7 @@ type SSPIConfig struct {
// FromDB fills up an SSPIConfig from serialized format.
func (cfg *SSPIConfig) FromDB(bs []byte) error {
json := jsoniter.ConfigCompatibleWithStandardLibrary
return json.Unmarshal(bs, cfg)
return jsonUnmarshalHandleDoubleEncode(bs, cfg)
}
// ToDB exports an SSPIConfig to a serialized format.

View File

@@ -17,7 +17,7 @@ import (
"code.gitea.io/gitea/modules/timeutil"
"code.gitea.io/gitea/modules/util"
"github.com/dgrijalva/jwt-go"
"github.com/golang-jwt/jwt"
uuid "github.com/google/uuid"
"golang.org/x/crypto/bcrypt"
"xorm.io/xorm"

View File

@@ -1215,7 +1215,7 @@ func ChangeRepositoryName(doer *User, repo *Repository, newRepoName string) (err
}
newRepoPath := RepoPath(repo.Owner.Name, newRepoName)
if err = os.Rename(repo.RepoPath(), newRepoPath); err != nil {
if err = util.Rename(repo.RepoPath(), newRepoPath); err != nil {
return fmt.Errorf("rename repository directory: %v", err)
}
@@ -1226,7 +1226,7 @@ func ChangeRepositoryName(doer *User, repo *Repository, newRepoName string) (err
return err
}
if isExist {
if err = os.Rename(wikiPath, WikiPath(repo.Owner.Name, newRepoName)); err != nil {
if err = util.Rename(wikiPath, WikiPath(repo.Owner.Name, newRepoName)); err != nil {
return fmt.Errorf("rename repository wiki: %v", err)
}
}

View File

@@ -210,13 +210,13 @@ func TransferOwnership(doer *User, newOwnerName string, repo *Repository) (err e
}
if repoRenamed {
if err := os.Rename(RepoPath(newOwnerName, repo.Name), RepoPath(oldOwnerName, repo.Name)); err != nil {
if err := util.Rename(RepoPath(newOwnerName, repo.Name), RepoPath(oldOwnerName, repo.Name)); err != nil {
log.Critical("Unable to move repository %s/%s directory from %s back to correct place %s: %v", oldOwnerName, repo.Name, RepoPath(newOwnerName, repo.Name), RepoPath(oldOwnerName, repo.Name), err)
}
}
if wikiRenamed {
if err := os.Rename(WikiPath(newOwnerName, repo.Name), WikiPath(oldOwnerName, repo.Name)); err != nil {
if err := util.Rename(WikiPath(newOwnerName, repo.Name), WikiPath(oldOwnerName, repo.Name)); err != nil {
log.Critical("Unable to move wiki for repository %s/%s directory from %s back to correct place %s: %v", oldOwnerName, repo.Name, WikiPath(newOwnerName, repo.Name), WikiPath(oldOwnerName, repo.Name), err)
}
}
@@ -358,7 +358,7 @@ func TransferOwnership(doer *User, newOwnerName string, repo *Repository) (err e
return fmt.Errorf("Failed to create dir %s: %v", dir, err)
}
if err := os.Rename(RepoPath(oldOwner.Name, repo.Name), RepoPath(newOwner.Name, repo.Name)); err != nil {
if err := util.Rename(RepoPath(oldOwner.Name, repo.Name), RepoPath(newOwner.Name, repo.Name)); err != nil {
return fmt.Errorf("rename repository directory: %v", err)
}
repoRenamed = true
@@ -370,7 +370,7 @@ func TransferOwnership(doer *User, newOwnerName string, repo *Repository) (err e
log.Error("Unable to check if %s exists. Error: %v", wikiPath, err)
return err
} else if isExist {
if err := os.Rename(wikiPath, WikiPath(newOwner.Name, repo.Name)); err != nil {
if err := util.Rename(wikiPath, WikiPath(newOwner.Name, repo.Name)); err != nil {
return fmt.Errorf("rename repository wiki: %v", err)
}
wikiRenamed = true

View File

@@ -28,8 +28,7 @@ type UnitConfig struct{}
// FromDB fills up a UnitConfig from serialized format.
func (cfg *UnitConfig) FromDB(bs []byte) error {
json := jsoniter.ConfigCompatibleWithStandardLibrary
return json.Unmarshal(bs, &cfg)
return jsonUnmarshalHandleDoubleEncode(bs, &cfg)
}
// ToDB exports a UnitConfig to a serialized format.
@@ -45,8 +44,7 @@ type ExternalWikiConfig struct {
// FromDB fills up a ExternalWikiConfig from serialized format.
func (cfg *ExternalWikiConfig) FromDB(bs []byte) error {
json := jsoniter.ConfigCompatibleWithStandardLibrary
return json.Unmarshal(bs, &cfg)
return jsonUnmarshalHandleDoubleEncode(bs, &cfg)
}
// ToDB exports a ExternalWikiConfig to a serialized format.
@@ -64,8 +62,7 @@ type ExternalTrackerConfig struct {
// FromDB fills up a ExternalTrackerConfig from serialized format.
func (cfg *ExternalTrackerConfig) FromDB(bs []byte) error {
json := jsoniter.ConfigCompatibleWithStandardLibrary
return json.Unmarshal(bs, &cfg)
return jsonUnmarshalHandleDoubleEncode(bs, &cfg)
}
// ToDB exports a ExternalTrackerConfig to a serialized format.
@@ -83,8 +80,7 @@ type IssuesConfig struct {
// FromDB fills up a IssuesConfig from serialized format.
func (cfg *IssuesConfig) FromDB(bs []byte) error {
json := jsoniter.ConfigCompatibleWithStandardLibrary
return json.Unmarshal(bs, &cfg)
return jsonUnmarshalHandleDoubleEncode(bs, &cfg)
}
// ToDB exports a IssuesConfig to a serialized format.
@@ -106,8 +102,7 @@ type PullRequestsConfig struct {
// FromDB fills up a PullRequestsConfig from serialized format.
func (cfg *PullRequestsConfig) FromDB(bs []byte) error {
json := jsoniter.ConfigCompatibleWithStandardLibrary
return json.Unmarshal(bs, &cfg)
return jsonUnmarshalHandleDoubleEncode(bs, &cfg)
}
// ToDB exports a PullRequestsConfig to a serialized format.

View File

@@ -834,7 +834,7 @@ func rewriteAllPublicKeys(e Engine) error {
}
t.Close()
return os.Rename(tmpPath, fPath)
return util.Rename(tmpPath, fPath)
}
// RegeneratePublicKeys regenerates the authorized_keys file
@@ -1316,7 +1316,7 @@ func rewriteAllPrincipalKeys(e Engine) error {
}
t.Close()
return os.Rename(tmpPath, fPath)
return util.Rename(tmpPath, fPath)
}
// ListPrincipalKeys returns a list of principals belongs to given user.

View File

@@ -1011,7 +1011,7 @@ func ChangeUserName(u *User, newUserName string) (err error) {
}
// Do not fail if directory does not exist
if err = os.Rename(UserPath(oldUserName), UserPath(newUserName)); err != nil && !os.IsNotExist(err) {
if err = util.Rename(UserPath(oldUserName), UserPath(newUserName)); err != nil && !os.IsNotExist(err) {
return fmt.Errorf("Rename user directory: %v", err)
}
@@ -1020,7 +1020,7 @@ func ChangeUserName(u *User, newUserName string) (err error) {
}
if err = sess.Commit(); err != nil {
if err2 := os.Rename(UserPath(newUserName), UserPath(oldUserName)); err2 != nil && !os.IsNotExist(err2) {
if err2 := util.Rename(UserPath(newUserName), UserPath(oldUserName)); err2 != nil && !os.IsNotExist(err2) {
log.Critical("Unable to rollback directory change during failed username change from: %s to: %s. DB Error: %v. Filesystem Error: %v", oldUserName, newUserName, err, err2)
return fmt.Errorf("failed to rollback directory change during failed username change from: %s to: %s. DB Error: %w. Filesystem Error: %v", oldUserName, newUserName, err, err2)
}

View File

@@ -22,6 +22,7 @@ import (
"net/http"
"time"
"code.gitea.io/gitea/modules/setting"
"code.gitea.io/gitea/modules/web/middleware"
"github.com/unknwon/com"
@@ -266,7 +267,12 @@ func Validate(ctx *Context, x CSRF) {
-1,
x.GetCookiePath(),
x.GetCookieDomain()) // FIXME: Do we need to set the Secure, httpOnly and SameSite values too?
x.Error(ctx.Resp)
if middleware.IsAPIPath(ctx.Req) {
x.Error(ctx.Resp)
return
}
ctx.Flash.Error(ctx.Tr("error.invalid_csrf"))
ctx.Redirect(setting.AppSubURL + "/")
}
return
}
@@ -277,10 +283,19 @@ func Validate(ctx *Context, x CSRF) {
-1,
x.GetCookiePath(),
x.GetCookieDomain()) // FIXME: Do we need to set the Secure, httpOnly and SameSite values too?
x.Error(ctx.Resp)
if middleware.IsAPIPath(ctx.Req) {
x.Error(ctx.Resp)
return
}
ctx.Flash.Error(ctx.Tr("error.invalid_csrf"))
ctx.Redirect(setting.AppSubURL + "/")
}
return
}
http.Error(ctx.Resp, "Bad Request: no CSRF token present", http.StatusBadRequest)
if middleware.IsAPIPath(ctx.Req) {
http.Error(ctx.Resp, "Bad Request: no CSRF token present", http.StatusBadRequest)
return
}
ctx.Flash.Error(ctx.Tr("error.missing_csrf"))
ctx.Redirect(setting.AppSubURL + "/")
}

View File

@@ -155,8 +155,8 @@ func ToCommit(repo *models.Repository, commit *git.Commit, userCache map[string]
URL: repo.APIURL() + "/git/commits/" + commit.ID.String(),
Author: &api.CommitUser{
Identity: api.Identity{
Name: commit.Committer.Name,
Email: commit.Committer.Email,
Name: commit.Author.Name,
Email: commit.Author.Email,
},
Date: commit.Author.When.Format(time.RFC3339),
},

View File

@@ -12,7 +12,7 @@ import (
"math/big"
"time"
"github.com/dgrijalva/jwt-go"
"github.com/golang-jwt/jwt"
)
// GetRandomString generate random string by specify chars.

View File

@@ -7,6 +7,7 @@ package git
import (
"bufio"
"bytes"
"context"
"io"
"math"
"strconv"
@@ -15,20 +16,24 @@ import (
// CatFileBatch opens git cat-file --batch in the provided repo and returns a stdin pipe, a stdout reader and cancel function
func CatFileBatch(repoPath string) (*io.PipeWriter, *bufio.Reader, func()) {
// Next feed the commits in order into cat-file --batch, followed by their trees and sub trees as necessary.
// We often want to feed the commits in order into cat-file --batch, followed by their trees and sub trees as necessary.
// so let's create a batch stdin and stdout
batchStdinReader, batchStdinWriter := io.Pipe()
batchStdoutReader, batchStdoutWriter := io.Pipe()
ctx, ctxCancel := context.WithCancel(DefaultContext)
closed := make(chan struct{})
cancel := func() {
_ = batchStdinReader.Close()
_ = batchStdinWriter.Close()
_ = batchStdoutReader.Close()
_ = batchStdoutWriter.Close()
ctxCancel()
<-closed
}
go func() {
stderr := strings.Builder{}
err := NewCommand("cat-file", "--batch").RunInDirFullPipeline(repoPath, batchStdoutWriter, &stderr, batchStdinReader)
err := NewCommandContext(ctx, "cat-file", "--batch").RunInDirFullPipeline(repoPath, batchStdoutWriter, &stderr, batchStdinReader)
if err != nil {
_ = batchStdoutWriter.CloseWithError(ConcatenateError(err, (&stderr).String()))
_ = batchStdinReader.CloseWithError(ConcatenateError(err, (&stderr).String()))
@@ -36,10 +41,11 @@ func CatFileBatch(repoPath string) (*io.PipeWriter, *bufio.Reader, func()) {
_ = batchStdoutWriter.Close()
_ = batchStdinReader.Close()
}
close(closed)
}()
// For simplicities sake we'll us a buffered reader to read from the cat-file --batch
batchReader := bufio.NewReader(batchStdoutReader)
batchReader := bufio.NewReaderSize(batchStdoutReader, 32*1024)
return batchStdinWriter, batchReader, cancel
}

View File

@@ -21,7 +21,7 @@ import (
"code.gitea.io/gitea/modules/setting"
"code.gitea.io/gitea/modules/storage"
"github.com/dgrijalva/jwt-go"
"github.com/golang-jwt/jwt"
jsoniter "github.com/json-iterator/go"
)

View File

@@ -177,7 +177,7 @@ func (log *FileLogger) DoRotate() error {
// close fd before rename
// Rename the file to its newfound home
if err = os.Rename(log.Filename, fname); err != nil {
if err = util.Rename(log.Filename, fname); err != nil {
return fmt.Errorf("Rotate: %v", err)
}

View File

@@ -334,40 +334,37 @@ func (ctx *postProcessCtx) postProcess(rawHTML []byte) ([]byte, error) {
_, _ = res.WriteString("</body></html>")
// parse the HTML
nodes, err := html.ParseFragment(res, nil)
node, err := html.Parse(res)
if err != nil {
return nil, &postProcessError{"invalid HTML", err}
}
for _, node := range nodes {
ctx.visitNode(node, true)
if node.Type == html.DocumentNode {
node = node.FirstChild
}
newNodes := make([]*html.Node, 0, len(nodes))
ctx.visitNode(node, true)
for _, node := range nodes {
if node.Data == "html" {
node = node.FirstChild
for node != nil && node.Data != "body" {
node = node.NextSibling
}
}
if node == nil {
continue
nodes := make([]*html.Node, 0, 5)
if node.Data == "html" {
node = node.FirstChild
for node != nil && node.Data != "body" {
node = node.NextSibling
}
}
if node != nil {
if node.Data == "body" {
child := node.FirstChild
for child != nil {
newNodes = append(newNodes, child)
nodes = append(nodes, child)
child = child.NextSibling
}
} else {
newNodes = append(newNodes, node)
nodes = append(nodes, node)
}
}
nodes = newNodes
// Create buffer in which the data will be placed again. We know that the
// length will be at least that of res; to spare a few alloc+copy, we
// reuse res, resetting its length to 0.
@@ -404,7 +401,7 @@ func (ctx *postProcessCtx) visitNode(node *html.Node, visitText bool) {
}
case html.ElementNode:
if node.Data == "img" {
for _, attr := range node.Attr {
for i, attr := range node.Attr {
if attr.Key != "src" {
continue
}
@@ -417,6 +414,7 @@ func (ctx *postProcessCtx) visitNode(node *html.Node, visitText bool) {
attr.Val = util.URLJoin(prefix, attr.Val)
}
node.Attr[i] = attr
}
} else if node.Data == "a" {
visitText = false

View File

@@ -124,13 +124,13 @@ func TestRender_links(t *testing.T) {
`<p><a href="http://www.example.com/wpstyle/?p=364" rel="nofollow">http://www.example.com/wpstyle/?p=364</a></p>`)
test(
"https://www.example.com/foo/?bar=baz&inga=42&quux",
`<p><a href="https://www.example.com/foo/?bar=baz&inga=42&quux" rel="nofollow">https://www.example.com/foo/?bar=baz&amp;inga=42&amp;quux</a></p>`)
`<p><a href="https://www.example.com/foo/?bar=baz&amp;inga=42&amp;quux" rel="nofollow">https://www.example.com/foo/?bar=baz&amp;inga=42&amp;quux</a></p>`)
test(
"http://142.42.1.1/",
`<p><a href="http://142.42.1.1/" rel="nofollow">http://142.42.1.1/</a></p>`)
test(
"https://github.com/go-gitea/gitea/?p=aaa/bbb.html#ccc-ddd",
`<p><a href="https://github.com/go-gitea/gitea/?p=aaa%2Fbbb.html#ccc-ddd" rel="nofollow">https://github.com/go-gitea/gitea/?p=aaa/bbb.html#ccc-ddd</a></p>`)
`<p><a href="https://github.com/go-gitea/gitea/?p=aaa/bbb.html#ccc-ddd" rel="nofollow">https://github.com/go-gitea/gitea/?p=aaa/bbb.html#ccc-ddd</a></p>`)
test(
"https://en.wikipedia.org/wiki/URL_(disambiguation)",
`<p><a href="https://en.wikipedia.org/wiki/URL_(disambiguation)" rel="nofollow">https://en.wikipedia.org/wiki/URL_(disambiguation)</a></p>`)
@@ -148,7 +148,7 @@ func TestRender_links(t *testing.T) {
`<p><a href="ftp://gitea.com/file.txt" rel="nofollow">ftp://gitea.com/file.txt</a></p>`)
test(
"magnet:?xt=urn:btih:5dee65101db281ac9c46344cd6b175cdcadabcde&dn=download",
`<p><a href="magnet:?xt=urn%3Abtih%3A5dee65101db281ac9c46344cd6b175cdcadabcde&dn=download" rel="nofollow">magnet:?xt=urn:btih:5dee65101db281ac9c46344cd6b175cdcadabcde&amp;dn=download</a></p>`)
`<p><a href="magnet:?xt=urn:btih:5dee65101db281ac9c46344cd6b175cdcadabcde&amp;dn=download" rel="nofollow">magnet:?xt=urn:btih:5dee65101db281ac9c46344cd6b175cdcadabcde&amp;dn=download</a></p>`)
// Test that should *not* be turned into URL
test(
@@ -384,6 +384,32 @@ func TestRender_ShortLinks(t *testing.T) {
`<p><a href="https://example.org" rel="nofollow">[[foobar]]</a></p>`)
}
func TestRender_RelativeImages(t *testing.T) {
setting.AppURL = AppURL
setting.AppSubURL = AppSubURL
tree := util.URLJoin(AppSubURL, "src", "master")
test := func(input, expected, expectedWiki string) {
buffer := markdown.RenderString(input, tree, localMetas)
assert.Equal(t, strings.TrimSpace(expected), strings.TrimSpace(buffer))
buffer = markdown.RenderWiki([]byte(input), setting.AppSubURL, localMetas)
assert.Equal(t, strings.TrimSpace(expectedWiki), strings.TrimSpace(buffer))
}
rawwiki := util.URLJoin(AppSubURL, "wiki", "raw")
mediatree := util.URLJoin(AppSubURL, "media", "master")
test(
`<img src="Link">`,
`<img src="`+util.URLJoin(mediatree, "Link")+`"/>`,
`<img src="`+util.URLJoin(rawwiki, "Link")+`"/>`)
test(
`<img src="./icon.png">`,
`<img src="`+util.URLJoin(mediatree, "icon.png")+`"/>`,
`<img src="`+util.URLJoin(rawwiki, "icon.png")+`"/>`)
}
func Test_ParseClusterFuzz(t *testing.T) {
setting.AppURL = AppURL
setting.AppSubURL = AppSubURL

View File

@@ -5,6 +5,7 @@
package references
import (
"bytes"
"net/url"
"regexp"
"strconv"
@@ -14,6 +15,8 @@ import (
"code.gitea.io/gitea/modules/log"
"code.gitea.io/gitea/modules/markup/mdstripper"
"code.gitea.io/gitea/modules/setting"
"github.com/yuin/goldmark/util"
)
var (
@@ -321,7 +324,7 @@ func FindRenderizableReferenceNumeric(content string, prOnly bool) (bool, *Rende
return false, nil
}
}
r := getCrossReference([]byte(content), match[2], match[3], false, prOnly)
r := getCrossReference(util.StringToReadOnlyBytes(content), match[2], match[3], false, prOnly)
if r == nil {
return false, nil
}
@@ -465,17 +468,16 @@ func findAllIssueReferencesBytes(content []byte, links []string) []*rawReference
}
func getCrossReference(content []byte, start, end int, fromLink bool, prOnly bool) *rawReference {
refid := string(content[start:end])
sep := strings.IndexAny(refid, "#!")
sep := bytes.IndexAny(content[start:end], "#!")
if sep < 0 {
return nil
}
isPull := refid[sep] == '!'
isPull := content[start+sep] == '!'
if prOnly && !isPull {
return nil
}
repo := refid[:sep]
issue := refid[sep+1:]
repo := string(content[start : start+sep])
issue := string(content[start+sep+1 : end])
index, err := strconv.ParseInt(issue, 10, 64)
if err != nil {
return nil

View File

@@ -96,7 +96,7 @@ func (l *LocalStorage) Save(path string, r io.Reader, size int64) (int64, error)
return 0, err
}
if err := os.Rename(tmp.Name(), p); err != nil {
if err := util.Rename(tmp.Name(), p); err != nil {
return 0, err
}

View File

@@ -6,10 +6,13 @@ package util
import (
"os"
"runtime"
"syscall"
"time"
)
const windowsSharingViolationError syscall.Errno = 32
// Remove removes the named file or (empty) directory with at most 5 attempts.
func Remove(name string) error {
var err error
@@ -25,6 +28,12 @@ func Remove(name string) error {
continue
}
if unwrapped == windowsSharingViolationError && runtime.GOOS == "windows" {
// try again
<-time.After(100 * time.Millisecond)
continue
}
if unwrapped == syscall.ENOENT {
// it's already gone
return nil
@@ -33,7 +42,7 @@ func Remove(name string) error {
return err
}
// RemoveAll removes the named file or (empty) directory with at most 5 attempts.Remove
// RemoveAll removes the named file or (empty) directory with at most 5 attempts.
func RemoveAll(name string) error {
var err error
for i := 0; i < 5; i++ {
@@ -48,6 +57,45 @@ func RemoveAll(name string) error {
continue
}
if unwrapped == windowsSharingViolationError && runtime.GOOS == "windows" {
// try again
<-time.After(100 * time.Millisecond)
continue
}
if unwrapped == syscall.ENOENT {
// it's already gone
return nil
}
}
return err
}
// Rename renames (moves) oldpath to newpath with at most 5 attempts.
func Rename(oldpath, newpath string) error {
var err error
for i := 0; i < 5; i++ {
err = os.Rename(oldpath, newpath)
if err == nil {
break
}
unwrapped := err.(*os.LinkError).Err
if unwrapped == syscall.EBUSY || unwrapped == syscall.ENOTEMPTY || unwrapped == syscall.EPERM || unwrapped == syscall.EMFILE || unwrapped == syscall.ENFILE {
// try again
<-time.After(100 * time.Millisecond)
continue
}
if unwrapped == windowsSharingViolationError && runtime.GOOS == "windows" {
// try again
<-time.After(100 * time.Millisecond)
continue
}
if i == 0 && os.IsNotExist(err) {
return err
}
if unwrapped == syscall.ENOENT {
// it's already gone
return nil

View File

@@ -96,6 +96,8 @@ error404 = The page you are trying to reach either <strong>does not exist</stron
[error]
occurred = An error has occurred
report_message = If you are sure this is a Gitea bug, please search for issue on <a href="https://github.com/go-gitea/gitea/issues">GitHub</a> and open new issue if necessary.
missing_csrf = Bad Request: no CSRF token present
invalid_csrf = Bad Request: Invalid CSRF token
[startpage]
app_desc = A painless, self-hosted Git service
@@ -719,6 +721,9 @@ mirror_address_desc = Put any required credentials in the Clone Authorization se
mirror_address_url_invalid = The provided url is invalid. You must escape all components of the url correctly.
mirror_address_protocol_invalid = The provided url is invalid. Only http(s):// or git:// locations can be mirrored from.
mirror_last_synced = Last Synchronized
mirror_password_placeholder = (Unchanged)
mirror_password_blank_placeholder = (Unset)
mirror_password_help = Change the username to erase a stored password.
watchers = Watchers
stargazers = Stargazers
forks = Forks

View File

@@ -986,10 +986,10 @@ func Routes() *web.Route {
Delete(reqToken(), reqOrgMembership(), org.ConcealMember)
})
m.Group("/teams", func() {
m.Combo("", reqToken()).Get(org.ListTeams).
Post(reqOrgOwnership(), bind(api.CreateTeamOption{}), org.CreateTeam)
m.Get("", org.ListTeams)
m.Post("", reqOrgOwnership(), bind(api.CreateTeamOption{}), org.CreateTeam)
m.Get("/search", org.SearchTeam)
}, reqOrgMembership())
}, reqToken(), reqOrgMembership())
m.Group("/labels", func() {
m.Get("", org.ListLabels)
m.Post("", reqToken(), reqOrgOwnership(), bind(api.CreateLabelOption{}), org.CreateLabel)

View File

@@ -17,7 +17,7 @@ func GetUserByParamsName(ctx *context.APIContext, name string) *models.User {
user, err := models.GetUserByName(username)
if err != nil {
if models.IsErrUserNotExist(err) {
if redirectUserID, err := models.LookupUserRedirect(username); err == nil {
if redirectUserID, err2 := models.LookupUserRedirect(username); err2 == nil {
context.RedirectToUser(ctx.Context, username, redirectUserID)
} else {
ctx.NotFound("GetUserByName", err)

View File

@@ -164,6 +164,12 @@ func SettingsPost(ctx *context.Context) {
}
}
oldUsername := mirror_service.Username(ctx.Repo.Mirror)
oldPassword := mirror_service.Password(ctx.Repo.Mirror)
if form.MirrorPassword == "" && form.MirrorUsername == oldUsername {
form.MirrorPassword = oldPassword
}
address, err := auth.ParseRemoteAddr(form.MirrorAddress, form.MirrorUsername, form.MirrorPassword)
if err == nil {
err = migrations.IsMigrateURLAllowed(address, ctx.User)

View File

@@ -21,7 +21,7 @@ import (
"code.gitea.io/gitea/modules/web"
"gitea.com/go-chi/binding"
"github.com/dgrijalva/jwt-go"
"github.com/golang-jwt/jwt"
)
const (

View File

@@ -193,16 +193,17 @@ func pushUpdates(optsList []*repo_module.PushUpdateOptions) error {
}
commits = repo_module.ListToPushCommits(l)
if err := repofiles.UpdateIssuesCommit(pusher, repo, commits.Commits, refName); err != nil {
log.Error("updateIssuesCommit: %v", err)
}
if len(commits.Commits) > setting.UI.FeedMaxCommitNum {
commits.Commits = commits.Commits[:setting.UI.FeedMaxCommitNum]
}
commits.CompareURL = repo.ComposeCompareURL(opts.OldCommitID, opts.NewCommitID)
notification.NotifyPushCommits(pusher, repo, opts, commits)
if err := repofiles.UpdateIssuesCommit(pusher, repo, commits.Commits, refName); err != nil {
log.Error("updateIssuesCommit: %v", err)
}
if err = models.RemoveDeletedBranch(repo.ID, branch); err != nil {
log.Error("models.RemoveDeletedBranch %s/%s failed: %v", repo.ID, branch, err)
}

View File

@@ -107,8 +107,9 @@
<input class="fake" type="password">
<div class="inline field {{if .Err_Auth}}error{{end}}">
<label for="mirror_password">{{.i18n.Tr "password"}}</label>
<input id="mirror_password" name="mirror_password" type="password" value="{{MirrorPassword .Mirror}}" {{if not .mirror_password}}data-need-clear="true"{{end}} autocomplete="off">
</div>
<input id="mirror_password" name="mirror_password" type="password" placeholder="{{if MirrorPassword .Mirror }}{{.i18n.Tr "repo.mirror_password_placeholder"}}{{else}}{{.i18n.Tr "repo.mirror_password_blank_placeholder"}}{{end}}" value="" {{if not .mirror_password}}data-need-clear="true"{{end}} autocomplete="off">
</div>
<p class="help">{{.i18n.Tr "repo.mirror_password_help"}}</p>
</div>
</div>

View File

@@ -177,6 +177,7 @@ func IsPrintableASCII(str string) bool
func IsRFC3339(str string) bool
func IsRFC3339WithoutZone(str string) bool
func IsRGBcolor(str string) bool
func IsRegex(str string) bool
func IsRequestURI(rawurl string) bool
func IsRequestURL(rawurl string) bool
func IsRipeMD128(str string) bool
@@ -203,6 +204,7 @@ func IsUUID(str string) bool
func IsUUIDv3(str string) bool
func IsUUIDv4(str string) bool
func IsUUIDv5(str string) bool
func IsULID(str string) bool
func IsUnixTime(str string) bool
func IsUpperCase(str string) bool
func IsVariableWidth(str string) bool
@@ -382,6 +384,7 @@ Here is a list of available validators for struct fields (validator - used funct
"rfc3339WithoutZone": IsRFC3339WithoutZone,
"ISO3166Alpha2": IsISO3166Alpha2,
"ISO3166Alpha3": IsISO3166Alpha3,
"ulid": IsULID,
```
Validators with parameters

View File

@@ -42,6 +42,8 @@ const (
SSN string = `^\d{3}[- ]?\d{2}[- ]?\d{4}$`
WinPath string = `^[a-zA-Z]:\\(?:[^\\/:*?"<>|\r\n]+\\)*[^\\/:*?"<>|\r\n]*$`
UnixPath string = `^(/[^/\x00]*)+/?$`
WinARPath string = `^(?:(?:[a-zA-Z]:|\\\\[a-z0-9_.$●-]+\\[a-z0-9_.$●-]+)\\|\\?[^\\/:*?"<>|\r\n]+\\?)(?:[^\\/:*?"<>|\r\n]+\\)*[^\\/:*?"<>|\r\n]*$`
UnixARPath string = `^((\.{0,2}/)?([^/\x00]*))+/?$`
Semver string = "^v?(?:0|[1-9]\\d*)\\.(?:0|[1-9]\\d*)\\.(?:0|[1-9]\\d*)(-(0|[1-9]\\d*|\\d*[a-zA-Z-][0-9a-zA-Z-]*)(\\.(0|[1-9]\\d*|\\d*[a-zA-Z-][0-9a-zA-Z-]*))*)?(\\+[0-9a-zA-Z-]+(\\.[0-9a-zA-Z-]+)*)?$"
tagName string = "valid"
hasLowerCase string = ".*[[:lower:]]"
@@ -50,6 +52,7 @@ const (
hasWhitespaceOnly string = "^[[:space:]]+$"
IMEI string = "^[0-9a-f]{14}$|^\\d{15}$|^\\d{18}$"
IMSI string = "^\\d{14,15}$"
E164 string = `^\+?[1-9]\d{1,14}$`
)
// Used by IsFilePath func
@@ -97,6 +100,8 @@ var (
rxSSN = regexp.MustCompile(SSN)
rxWinPath = regexp.MustCompile(WinPath)
rxUnixPath = regexp.MustCompile(UnixPath)
rxARWinPath = regexp.MustCompile(WinARPath)
rxARUnixPath = regexp.MustCompile(UnixARPath)
rxSemver = regexp.MustCompile(Semver)
rxHasLowerCase = regexp.MustCompile(hasLowerCase)
rxHasUpperCase = regexp.MustCompile(hasUpperCase)
@@ -104,4 +109,5 @@ var (
rxHasWhitespaceOnly = regexp.MustCompile(hasWhitespaceOnly)
rxIMEI = regexp.MustCompile(IMEI)
rxIMSI = regexp.MustCompile(IMSI)
rxE164 = regexp.MustCompile(E164)
)

View File

@@ -165,6 +165,7 @@ var TagMap = map[string]Validator{
"ISO3166Alpha3": IsISO3166Alpha3,
"ISO4217": IsISO4217,
"IMEI": IsIMEI,
"ulid": IsULID,
}
// ISO3166Entry stores country codes

View File

@@ -361,9 +361,96 @@ func IsUUID(str string) bool {
return rxUUID.MatchString(str)
}
// Byte to index table for O(1) lookups when unmarshaling.
// We use 0xFF as sentinel value for invalid indexes.
var ulidDec = [...]byte{
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x00, 0x01,
0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0xFF, 0xFF,
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x0A, 0x0B, 0x0C, 0x0D, 0x0E,
0x0F, 0x10, 0x11, 0xFF, 0x12, 0x13, 0xFF, 0x14, 0x15, 0xFF,
0x16, 0x17, 0x18, 0x19, 0x1A, 0xFF, 0x1B, 0x1C, 0x1D, 0x1E,
0x1F, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x0A, 0x0B, 0x0C,
0x0D, 0x0E, 0x0F, 0x10, 0x11, 0xFF, 0x12, 0x13, 0xFF, 0x14,
0x15, 0xFF, 0x16, 0x17, 0x18, 0x19, 0x1A, 0xFF, 0x1B, 0x1C,
0x1D, 0x1E, 0x1F, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
}
// EncodedSize is the length of a text encoded ULID.
const ulidEncodedSize = 26
// IsULID checks if the string is a ULID.
//
// Implementation got from:
// https://github.com/oklog/ulid (Apache-2.0 License)
//
func IsULID(str string) bool {
// Check if a base32 encoded ULID is the right length.
if len(str) != ulidEncodedSize {
return false
}
// Check if all the characters in a base32 encoded ULID are part of the
// expected base32 character set.
if ulidDec[str[0]] == 0xFF ||
ulidDec[str[1]] == 0xFF ||
ulidDec[str[2]] == 0xFF ||
ulidDec[str[3]] == 0xFF ||
ulidDec[str[4]] == 0xFF ||
ulidDec[str[5]] == 0xFF ||
ulidDec[str[6]] == 0xFF ||
ulidDec[str[7]] == 0xFF ||
ulidDec[str[8]] == 0xFF ||
ulidDec[str[9]] == 0xFF ||
ulidDec[str[10]] == 0xFF ||
ulidDec[str[11]] == 0xFF ||
ulidDec[str[12]] == 0xFF ||
ulidDec[str[13]] == 0xFF ||
ulidDec[str[14]] == 0xFF ||
ulidDec[str[15]] == 0xFF ||
ulidDec[str[16]] == 0xFF ||
ulidDec[str[17]] == 0xFF ||
ulidDec[str[18]] == 0xFF ||
ulidDec[str[19]] == 0xFF ||
ulidDec[str[20]] == 0xFF ||
ulidDec[str[21]] == 0xFF ||
ulidDec[str[22]] == 0xFF ||
ulidDec[str[23]] == 0xFF ||
ulidDec[str[24]] == 0xFF ||
ulidDec[str[25]] == 0xFF {
return false
}
// Check if the first character in a base32 encoded ULID will overflow. This
// happens because the base32 representation encodes 130 bits, while the
// ULID is only 128 bits.
//
// See https://github.com/oklog/ulid/issues/9 for details.
if str[0] > '7' {
return false
}
return true
}
// IsCreditCard checks if the string is a credit card.
func IsCreditCard(str string) bool {
sanitized := notNumberRegexp.ReplaceAllString(str, "")
sanitized := whiteSpacesAndMinus.ReplaceAllString(str, "")
if !rxCreditCard.MatchString(sanitized) {
return false
}
@@ -509,6 +596,27 @@ func IsFilePath(str string) (bool, int) {
return false, Unknown
}
//IsWinFilePath checks both relative & absolute paths in Windows
func IsWinFilePath(str string) bool {
if rxARWinPath.MatchString(str) {
//check windows path limit see:
// http://msdn.microsoft.com/en-us/library/aa365247(VS.85).aspx#maxpath
if len(str[3:]) > 32767 {
return false
}
return true
}
return false
}
//IsUnixFilePath checks both relative & absolute paths in Unix
func IsUnixFilePath(str string) bool {
if rxARUnixPath.MatchString(str) {
return true
}
return false
}
// IsDataURI checks if a string is base64 encoded data URI such as an image
func IsDataURI(str string) bool {
dataURI := strings.Split(str, ",")
@@ -586,11 +694,13 @@ func IsHash(str string, algorithm string) bool {
len = "40"
} else if algo == "tiger192" {
len = "48"
} else if algo == "sha256" {
} else if algo == "sha3-224" {
len = "56"
} else if algo == "sha256" || algo == "sha3-256" {
len = "64"
} else if algo == "sha384" {
} else if algo == "sha384" || algo == "sha3-384" {
len = "96"
} else if algo == "sha512" {
} else if algo == "sha512" || algo == "sha3-512" {
len = "128"
} else {
return false
@@ -599,6 +709,26 @@ func IsHash(str string, algorithm string) bool {
return Matches(str, "^[a-f0-9]{"+len+"}$")
}
// IsSHA3224 checks is a string is a SHA3-224 hash. Alias for `IsHash(str, "sha3-224")`
func IsSHA3224(str string) bool {
return IsHash(str, "sha3-224")
}
// IsSHA3256 checks is a string is a SHA3-256 hash. Alias for `IsHash(str, "sha3-256")`
func IsSHA3256(str string) bool {
return IsHash(str, "sha3-256")
}
// IsSHA3384 checks is a string is a SHA3-384 hash. Alias for `IsHash(str, "sha3-384")`
func IsSHA3384(str string) bool {
return IsHash(str, "sha3-384")
}
// IsSHA3512 checks is a string is a SHA3-512 hash. Alias for `IsHash(str, "sha3-512")`
func IsSHA3512(str string) bool {
return IsHash(str, "sha3-512")
}
// IsSHA512 checks is a string is a SHA512 hash. Alias for `IsHash(str, "sha512")`
func IsSHA512(str string) bool {
return IsHash(str, "sha512")
@@ -819,6 +949,14 @@ func IsRsaPublicKey(str string, keylen int) bool {
return bitlen == int(keylen)
}
// IsRegex checks if a give string is a valid regex with RE2 syntax or not
func IsRegex(str string) bool {
if _, err := regexp.Compile(str); err == nil {
return true
}
return false
}
func toJSONName(tag string) string {
if tag == "" {
return ""
@@ -1625,3 +1763,7 @@ func (sv stringValues) Len() int { return len(sv) }
func (sv stringValues) Swap(i, j int) { sv[i], sv[j] = sv[j], sv[i] }
func (sv stringValues) Less(i, j int) bool { return sv.get(i) < sv.get(j) }
func (sv stringValues) get(i int) string { return sv[i].String() }
func IsE164(str string) bool {
return rxE164.MatchString(str)
}

View File

@@ -1,13 +0,0 @@
language: go
script:
- go vet ./...
- go test -v ./...
go:
- 1.3
- 1.4
- 1.5
- 1.6
- 1.7
- tip

View File

@@ -1,97 +0,0 @@
## Migration Guide from v2 -> v3
Version 3 adds several new, frequently requested features. To do so, it introduces a few breaking changes. We've worked to keep these as minimal as possible. This guide explains the breaking changes and how you can quickly update your code.
### `Token.Claims` is now an interface type
The most requested feature from the 2.0 verison of this library was the ability to provide a custom type to the JSON parser for claims. This was implemented by introducing a new interface, `Claims`, to replace `map[string]interface{}`. We also included two concrete implementations of `Claims`: `MapClaims` and `StandardClaims`.
`MapClaims` is an alias for `map[string]interface{}` with built in validation behavior. It is the default claims type when using `Parse`. The usage is unchanged except you must type cast the claims property.
The old example for parsing a token looked like this..
```go
if token, err := jwt.Parse(tokenString, keyLookupFunc); err == nil {
fmt.Printf("Token for user %v expires %v", token.Claims["user"], token.Claims["exp"])
}
```
is now directly mapped to...
```go
if token, err := jwt.Parse(tokenString, keyLookupFunc); err == nil {
claims := token.Claims.(jwt.MapClaims)
fmt.Printf("Token for user %v expires %v", claims["user"], claims["exp"])
}
```
`StandardClaims` is designed to be embedded in your custom type. You can supply a custom claims type with the new `ParseWithClaims` function. Here's an example of using a custom claims type.
```go
type MyCustomClaims struct {
User string
*StandardClaims
}
if token, err := jwt.ParseWithClaims(tokenString, &MyCustomClaims{}, keyLookupFunc); err == nil {
claims := token.Claims.(*MyCustomClaims)
fmt.Printf("Token for user %v expires %v", claims.User, claims.StandardClaims.ExpiresAt)
}
```
### `ParseFromRequest` has been moved
To keep this library focused on the tokens without becoming overburdened with complex request processing logic, `ParseFromRequest` and its new companion `ParseFromRequestWithClaims` have been moved to a subpackage, `request`. The method signatues have also been augmented to receive a new argument: `Extractor`.
`Extractors` do the work of picking the token string out of a request. The interface is simple and composable.
This simple parsing example:
```go
if token, err := jwt.ParseFromRequest(tokenString, req, keyLookupFunc); err == nil {
fmt.Printf("Token for user %v expires %v", token.Claims["user"], token.Claims["exp"])
}
```
is directly mapped to:
```go
if token, err := request.ParseFromRequest(req, request.OAuth2Extractor, keyLookupFunc); err == nil {
claims := token.Claims.(jwt.MapClaims)
fmt.Printf("Token for user %v expires %v", claims["user"], claims["exp"])
}
```
There are several concrete `Extractor` types provided for your convenience:
* `HeaderExtractor` will search a list of headers until one contains content.
* `ArgumentExtractor` will search a list of keys in request query and form arguments until one contains content.
* `MultiExtractor` will try a list of `Extractors` in order until one returns content.
* `AuthorizationHeaderExtractor` will look in the `Authorization` header for a `Bearer` token.
* `OAuth2Extractor` searches the places an OAuth2 token would be specified (per the spec): `Authorization` header and `access_token` argument
* `PostExtractionFilter` wraps an `Extractor`, allowing you to process the content before it's parsed. A simple example is stripping the `Bearer ` text from a header
### RSA signing methods no longer accept `[]byte` keys
Due to a [critical vulnerability](https://auth0.com/blog/2015/03/31/critical-vulnerabilities-in-json-web-token-libraries/), we've decided the convenience of accepting `[]byte` instead of `rsa.PublicKey` or `rsa.PrivateKey` isn't worth the risk of misuse.
To replace this behavior, we've added two helper methods: `ParseRSAPrivateKeyFromPEM(key []byte) (*rsa.PrivateKey, error)` and `ParseRSAPublicKeyFromPEM(key []byte) (*rsa.PublicKey, error)`. These are just simple helpers for unpacking PEM encoded PKCS1 and PKCS8 keys. If your keys are encoded any other way, all you need to do is convert them to the `crypto/rsa` package's types.
```go
func keyLookupFunc(*Token) (interface{}, error) {
// Don't forget to validate the alg is what you expect:
if _, ok := token.Method.(*jwt.SigningMethodRSA); !ok {
return nil, fmt.Errorf("Unexpected signing method: %v", token.Header["alg"])
}
// Look up key
key, err := lookupPublicKey(token.Header["kid"])
if err != nil {
return nil, err
}
// Unpack key from PEM encoded PKCS8
return jwt.ParseRSAPublicKeyFromPEM(key)
}
```

View File

@@ -1,4 +1,4 @@
.DS_Store
bin
.idea/

View File

@@ -1,4 +1,5 @@
Copyright (c) 2012 Dave Grijalva
Copyright (c) 2021 golang-jwt maintainers
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

22
vendor/github.com/golang-jwt/jwt/MIGRATION_GUIDE.md generated vendored Normal file
View File

@@ -0,0 +1,22 @@
## Migration Guide (v3.2.1)
Starting from [v3.2.1](https://github.com/golang-jwt/jwt/releases/tag/v3.2.1]), the import path has changed from `github.com/dgrijalva/jwt-go` to `github.com/golang-jwt/jwt`. Future releases will be using the `github.com/golang-jwt/jwt` import path and continue the existing versioning scheme of `v3.x.x+incompatible`. Backwards-compatible patches and fixes will be done on the `v3` release branch, where as new build-breaking features will be developed in a `v4` release, possibly including a SIV-style import path.
### go.mod replacement
In a first step, the easiest way is to use `go mod edit` to issue a replacement.
```
go mod edit -replace github.com/dgrijalva/jwt-go=github.com/golang-jwt/jwt@v3.2.1+incompatible
go mod tidy
```
This will still keep the old import path in your code but replace it with the new package and also introduce a new indirect dependency to `github.com/golang-jwt/jwt`. Try to compile your project; it should still work.
### Cleanup
If your code still consistently builds, you can replace all occurences of `github.com/dgrijalva/jwt-go` with `github.com/golang-jwt/jwt`, either manually or by using tools such as `sed`. Finally, the `replace` directive in the `go.mod` file can be removed.
## Older releases (before v3.2.0)
The original migration guide for older releases can be found at https://github.com/dgrijalva/jwt-go/blob/master/MIGRATION_GUIDE.md.

View File

@@ -1,25 +1,34 @@
# jwt-go
[![Build Status](https://travis-ci.org/dgrijalva/jwt-go.svg?branch=master)](https://travis-ci.org/dgrijalva/jwt-go)
[![GoDoc](https://godoc.org/github.com/dgrijalva/jwt-go?status.svg)](https://godoc.org/github.com/dgrijalva/jwt-go)
[![build](https://github.com/golang-jwt/jwt/actions/workflows/build.yml/badge.svg)](https://github.com/golang-jwt/jwt/actions/workflows/build.yml)
[![Go Reference](https://pkg.go.dev/badge/github.com/golang-jwt/jwt.svg)](https://pkg.go.dev/github.com/golang-jwt/jwt)
A [go](http://www.golang.org) (or 'golang' for search engine friendliness) implementation of [JSON Web Tokens](http://self-issued.info/docs/draft-ietf-oauth-json-web-token.html)
A [go](http://www.golang.org) (or 'golang' for search engine friendliness) implementation of [JSON Web Tokens](https://datatracker.ietf.org/doc/html/rfc7519).
**NEW VERSION COMING:** There have been a lot of improvements suggested since the version 3.0.0 released in 2016. I'm working now on cutting two different releases: 3.2.0 will contain any non-breaking changes or enhancements. 4.0.0 will follow shortly which will include breaking changes. See the 4.0.0 milestone to get an idea of what's coming. If you have other ideas, or would like to participate in 4.0.0, now's the time. If you depend on this library and don't want to be interrupted, I recommend you use your dependency mangement tool to pin to version 3.
**IMPORT PATH CHANGE:** Starting from [v3.2.1](https://github.com/golang-jwt/jwt/releases/tag/v3.2.1), the import path has changed from `github.com/dgrijalva/jwt-go` to `github.com/golang-jwt/jwt`. After the original author of the library suggested migrating the maintenance of `jwt-go`, a dedicated team of open source maintainers decided to clone the existing library into this repository. See [dgrijalva/jwt-go#462](https://github.com/dgrijalva/jwt-go/issues/462) for a detailed discussion on this topic.
**SECURITY NOTICE:** Some older versions of Go have a security issue in the cryotp/elliptic. Recommendation is to upgrade to at least 1.8.3. See issue #216 for more detail.
Future releases will be using the `github.com/golang-jwt/jwt` import path and continue the existing versioning scheme of `v3.x.x+incompatible`. Backwards-compatible patches and fixes will be done on the `v3` release branch, where as new build-breaking features will be developed in a `v4` release, possibly including a SIV-style import path.
**SECURITY NOTICE:** It's important that you [validate the `alg` presented is what you expect](https://auth0.com/blog/2015/03/31/critical-vulnerabilities-in-json-web-token-libraries/). This library attempts to make it easy to do the right thing by requiring key types match the expected alg, but you should take the extra step to verify it in your usage. See the examples provided.
**SECURITY NOTICE:** Some older versions of Go have a security issue in the crypto/elliptic. Recommendation is to upgrade to at least 1.15 See issue [dgrijalva/jwt-go#216](https://github.com/dgrijalva/jwt-go/issues/216) for more detail.
**SECURITY NOTICE:** It's important that you [validate the `alg` presented is what you expect](https://auth0.com/blog/critical-vulnerabilities-in-json-web-token-libraries/). This library attempts to make it easy to do the right thing by requiring key types match the expected alg, but you should take the extra step to verify it in your usage. See the examples provided.
### Supported Go versions
Our support of Go versions is aligned with Go's [version release policy](https://golang.org/doc/devel/release#policy).
So we will support a major version of Go until there are two newer major releases.
We no longer support building jwt-go with unsupported Go versions, as these contain security vulnerabilities
which will not be fixed.
## What the heck is a JWT?
JWT.io has [a great introduction](https://jwt.io/introduction) to JSON Web Tokens.
In short, it's a signed JSON object that does something useful (for example, authentication). It's commonly used for `Bearer` tokens in Oauth 2. A token is made of three parts, separated by `.`'s. The first two parts are JSON objects, that have been [base64url](http://tools.ietf.org/html/rfc4648) encoded. The last part is the signature, encoded the same way.
In short, it's a signed JSON object that does something useful (for example, authentication). It's commonly used for `Bearer` tokens in Oauth 2. A token is made of three parts, separated by `.`'s. The first two parts are JSON objects, that have been [base64url](https://datatracker.ietf.org/doc/html/rfc4648) encoded. The last part is the signature, encoded the same way.
The first part is called the header. It contains the necessary information for verifying the last part, the signature. For example, which encryption method was used for signing and what key was used.
The part in the middle is the interesting bit. It's called the Claims and contains the actual stuff you care about. Refer to [the RFC](http://self-issued.info/docs/draft-jones-json-web-token.html) for information about reserved keys and the proper way to add your own.
The part in the middle is the interesting bit. It's called the Claims and contains the actual stuff you care about. Refer to [RFC 7519](https://datatracker.ietf.org/doc/html/rfc7519) for information about reserved keys and the proper way to add your own.
## What's in the box?
@@ -27,31 +36,31 @@ This library supports the parsing and verification as well as the generation and
## Examples
See [the project documentation](https://godoc.org/github.com/dgrijalva/jwt-go) for examples of usage:
See [the project documentation](https://pkg.go.dev/github.com/golang-jwt/jwt) for examples of usage:
* [Simple example of parsing and validating a token](https://godoc.org/github.com/dgrijalva/jwt-go#example-Parse--Hmac)
* [Simple example of building and signing a token](https://godoc.org/github.com/dgrijalva/jwt-go#example-New--Hmac)
* [Directory of Examples](https://godoc.org/github.com/dgrijalva/jwt-go#pkg-examples)
* [Simple example of parsing and validating a token](https://pkg.go.dev/github.com/golang-jwt/jwt#example-Parse-Hmac)
* [Simple example of building and signing a token](https://pkg.go.dev/github.com/golang-jwt/jwt#example-New-Hmac)
* [Directory of Examples](https://pkg.go.dev/github.com/golang-jwt/jwt#pkg-examples)
## Extensions
This library publishes all the necessary components for adding your own signing methods. Simply implement the `SigningMethod` interface and register a factory method using `RegisterSigningMethod`.
Here's an example of an extension that integrates with the Google App Engine signing tools: https://github.com/someone1/gcp-jwt-go
Here's an example of an extension that integrates with multiple Google Cloud Platform signing tools (AppEngine, IAM API, Cloud KMS): https://github.com/someone1/gcp-jwt-go
## Compliance
This library was last reviewed to comply with [RTF 7519](http://www.rfc-editor.org/info/rfc7519) dated May 2015 with a few notable differences:
This library was last reviewed to comply with [RTF 7519](https://datatracker.ietf.org/doc/html/rfc7519) dated May 2015 with a few notable differences:
* In order to protect against accidental use of [Unsecured JWTs](http://self-issued.info/docs/draft-ietf-oauth-json-web-token.html#UnsecuredJWT), tokens using `alg=none` will only be accepted if the constant `jwt.UnsafeAllowNoneSignatureType` is provided as the key.
* In order to protect against accidental use of [Unsecured JWTs](https://datatracker.ietf.org/doc/html/rfc7519#section-6), tokens using `alg=none` will only be accepted if the constant `jwt.UnsafeAllowNoneSignatureType` is provided as the key.
## Project Status & Versioning
This library is considered production ready. Feedback and feature requests are appreciated. The API should be considered stable. There should be very few backwards-incompatible changes outside of major version updates (and only with good reason).
This project uses [Semantic Versioning 2.0.0](http://semver.org). Accepted pull requests will land on `master`. Periodically, versions will be tagged from `master`. You can find all the releases on [the project releases page](https://github.com/dgrijalva/jwt-go/releases).
This project uses [Semantic Versioning 2.0.0](http://semver.org). Accepted pull requests will land on `main`. Periodically, versions will be tagged from `main`. You can find all the releases on [the project releases page](https://github.com/golang-jwt/jwt/releases).
While we try to make it obvious when we make breaking changes, there isn't a great mechanism for pushing announcements out to users. You may want to use this alternative package include: `gopkg.in/dgrijalva/jwt-go.v3`. It will do the right thing WRT semantic versioning.
While we try to make it obvious when we make breaking changes, there isn't a great mechanism for pushing announcements out to users. You may want to use this alternative package include: `gopkg.in/golang-jwt/jwt.v3`. It will do the right thing WRT semantic versioning.
**BREAKING CHANGES:***
* Version 3.0.0 includes _a lot_ of changes from the 2.x line, including a few that break the API. We've tried to break as few things as possible, so there should just be a few type signature changes. A full list of breaking changes is available in `VERSION_HISTORY.md`. See `MIGRATION_GUIDE.md` for more information on updating your code.
@@ -79,9 +88,9 @@ Asymmetric signing methods, such as RSA, use different keys for signing and veri
Each signing method expects a different object type for its signing keys. See the package documentation for details. Here are the most common ones:
* The [HMAC signing method](https://godoc.org/github.com/dgrijalva/jwt-go#SigningMethodHMAC) (`HS256`,`HS384`,`HS512`) expect `[]byte` values for signing and validation
* The [RSA signing method](https://godoc.org/github.com/dgrijalva/jwt-go#SigningMethodRSA) (`RS256`,`RS384`,`RS512`) expect `*rsa.PrivateKey` for signing and `*rsa.PublicKey` for validation
* The [ECDSA signing method](https://godoc.org/github.com/dgrijalva/jwt-go#SigningMethodECDSA) (`ES256`,`ES384`,`ES512`) expect `*ecdsa.PrivateKey` for signing and `*ecdsa.PublicKey` for validation
* The [HMAC signing method](https://pkg.go.dev/github.com/golang-jwt/jwt#SigningMethodHMAC) (`HS256`,`HS384`,`HS512`) expect `[]byte` values for signing and validation
* The [RSA signing method](https://pkg.go.dev/github.com/golang-jwt/jwt#SigningMethodRSA) (`RS256`,`RS384`,`RS512`) expect `*rsa.PrivateKey` for signing and `*rsa.PublicKey` for validation
* The [ECDSA signing method](https://pkg.go.dev/github.com/golang-jwt/jwt#SigningMethodECDSA) (`ES256`,`ES384`,`ES512`) expect `*ecdsa.PrivateKey` for signing and `*ecdsa.PublicKey` for validation
### JWT and OAuth
@@ -93,8 +102,12 @@ Without going too far down the rabbit hole, here's a description of the interact
* OAuth defines several options for passing around authentication data. One popular method is called a "bearer token". A bearer token is simply a string that _should_ only be held by an authenticated user. Thus, simply presenting this token proves your identity. You can probably derive from here why a JWT might make a good bearer token.
* Because bearer tokens are used for authentication, it's important they're kept secret. This is why transactions that use bearer tokens typically happen over SSL.
### Troubleshooting
This library uses descriptive error messages whenever possible. If you are not getting the expected result, have a look at the errors. The most common place people get stuck is providing the correct type of key to the parser. See the above section on signing methods and key types.
## More
Documentation can be found [on godoc.org](http://godoc.org/github.com/dgrijalva/jwt-go).
Documentation can be found [on pkg.go.dev](https://pkg.go.dev/github.com/golang-jwt/jwt).
The command line utility included in this project (cmd/jwt) provides a straightforward example of token creation and parsing as well as a useful tool for debugging your own integration. You'll also find several implementation examples in the documentation.

View File

@@ -1,5 +1,23 @@
## `jwt-go` Version History
#### 3.2.2-go1.14
Whilst go1.14 is not officially supported upstream anymore there are a few projects which require go1.14 compilation support. This forked release simply contains a workaround for go1.14.
#### 3.2.2
* Starting from this release, we are adopting the policy to support the most 2 recent versions of Go currently available. By the time of this release, this is Go 1.15 and 1.16 ([#28](https://github.com/golang-jwt/jwt/pull/28)).
* Fixed a potential issue that could occur when the verification of `exp`, `iat` or `nbf` was not required and contained invalid contents, i.e. non-numeric/date. Thanks for @thaJeztah for making us aware of that and @giorgos-f3 for originally reporting it to the formtech fork ([#40](https://github.com/golang-jwt/jwt/pull/40)).
* Added support for EdDSA / ED25519 ([#36](https://github.com/golang-jwt/jwt/pull/36)).
* Optimized allocations ([#33](https://github.com/golang-jwt/jwt/pull/33)).
#### 3.2.1
* **Import Path Change**: See MIGRATION_GUIDE.md for tips on updating your code
* Changed the import path from `github.com/dgrijalva/jwt-go` to `github.com/golang-jwt/jwt`
* Fixed type confusing issue between `string` and `[]string` in `VerifyAudience` ([#12](https://github.com/golang-jwt/jwt/pull/12)). This fixes CVE-2020-26160
#### 3.2.0
* Added method `ParseUnverified` to allow users to split up the tasks of parsing and validation
@@ -115,4 +133,4 @@ It is likely the only integration change required here will be to change `func(t
* First versioned release
* API stabilized
* Supports creating, signing, parsing, and validating JWT tokens
* Supports RS256 and HS256 signing methods
* Supports RS256 and HS256 signing methods

View File

@@ -35,18 +35,18 @@ func (c StandardClaims) Valid() error {
// The claims below are optional, by default, so if they are set to the
// default value in Go, let's not fail the verification for them.
if c.VerifyExpiresAt(now, false) == false {
if !c.VerifyExpiresAt(now, false) {
delta := time.Unix(now, 0).Sub(time.Unix(c.ExpiresAt, 0))
vErr.Inner = fmt.Errorf("token is expired by %v", delta)
vErr.Errors |= ValidationErrorExpired
}
if c.VerifyIssuedAt(now, false) == false {
if !c.VerifyIssuedAt(now, false) {
vErr.Inner = fmt.Errorf("Token used before issued")
vErr.Errors |= ValidationErrorIssuedAt
}
if c.VerifyNotBefore(now, false) == false {
if !c.VerifyNotBefore(now, false) {
vErr.Inner = fmt.Errorf("token is not valid yet")
vErr.Errors |= ValidationErrorNotValidYet
}
@@ -61,7 +61,7 @@ func (c StandardClaims) Valid() error {
// Compares the aud claim against cmp.
// If required is false, this method will return true if the value matches or is unset
func (c *StandardClaims) VerifyAudience(cmp string, req bool) bool {
return verifyAud(c.Audience, cmp, req)
return verifyAud([]string{c.Audience}, cmp, req)
}
// Compares the exp claim against cmp.
@@ -90,15 +90,27 @@ func (c *StandardClaims) VerifyNotBefore(cmp int64, req bool) bool {
// ----- helpers
func verifyAud(aud string, cmp string, required bool) bool {
if aud == "" {
func verifyAud(aud []string, cmp string, required bool) bool {
if len(aud) == 0 {
return !required
}
if subtle.ConstantTimeCompare([]byte(aud), []byte(cmp)) != 0 {
return true
} else {
return false
// use a var here to keep constant time compare when looping over a number of claims
result := false
var stringClaims string
for _, a := range aud {
if subtle.ConstantTimeCompare([]byte(a), []byte(cmp)) != 0 {
result = true
}
stringClaims = stringClaims + a
}
// case where "" is sent in one or many aud claims
if len(stringClaims) == 0 {
return !required
}
return result
}
func verifyExp(exp int64, now int64, required bool) bool {

View File

@@ -3,7 +3,6 @@ package jwt
import (
"crypto"
"crypto/ecdsa"
"crypto/rand"
"errors"
"math/big"
)
@@ -88,61 +87,9 @@ func (m *SigningMethodECDSA) Verify(signingString, signature string, key interfa
hasher.Write([]byte(signingString))
// Verify the signature
if verifystatus := ecdsa.Verify(ecdsaKey, hasher.Sum(nil), r, s); verifystatus == true {
if verifystatus := ecdsa.Verify(ecdsaKey, hasher.Sum(nil), r, s); verifystatus {
return nil
} else {
return ErrECDSAVerification
}
}
// Implements the Sign method from SigningMethod
// For this signing method, key must be an ecdsa.PrivateKey struct
func (m *SigningMethodECDSA) Sign(signingString string, key interface{}) (string, error) {
// Get the key
var ecdsaKey *ecdsa.PrivateKey
switch k := key.(type) {
case *ecdsa.PrivateKey:
ecdsaKey = k
default:
return "", ErrInvalidKeyType
}
// Create the hasher
if !m.Hash.Available() {
return "", ErrHashUnavailable
}
hasher := m.Hash.New()
hasher.Write([]byte(signingString))
// Sign the string and return r, s
if r, s, err := ecdsa.Sign(rand.Reader, ecdsaKey, hasher.Sum(nil)); err == nil {
curveBits := ecdsaKey.Curve.Params().BitSize
if m.CurveBits != curveBits {
return "", ErrInvalidKey
}
keyBytes := curveBits / 8
if curveBits%8 > 0 {
keyBytes += 1
}
// We serialize the outpus (r and s) into big-endian byte arrays and pad
// them with zeros on the left to make sure the sizes work out. Both arrays
// must be keyBytes long, and the output must be 2*keyBytes long.
rBytes := r.Bytes()
rBytesPadded := make([]byte, keyBytes)
copy(rBytesPadded[keyBytes-len(rBytes):], rBytes)
sBytes := s.Bytes()
sBytesPadded := make([]byte, keyBytes)
copy(sBytesPadded[keyBytes-len(sBytes):], sBytes)
out := append(rBytesPadded, sBytesPadded...)
return EncodeSegment(out), nil
} else {
return "", err
}
return ErrECDSAVerification
}

99
vendor/github.com/golang-jwt/jwt/ecdsa_go1.14.go generated vendored Normal file
View File

@@ -0,0 +1,99 @@
//+build !go1.15
package jwt
import (
"crypto/ecdsa"
"crypto/rand"
"math/big"
"math/bits"
)
// Implements the Sign method from SigningMethod
// For this signing method, key must be an ecdsa.PrivateKey struct
func (m *SigningMethodECDSA) Sign(signingString string, key interface{}) (string, error) {
// Get the key
var ecdsaKey *ecdsa.PrivateKey
switch k := key.(type) {
case *ecdsa.PrivateKey:
ecdsaKey = k
default:
return "", ErrInvalidKeyType
}
// Create the hasher
if !m.Hash.Available() {
return "", ErrHashUnavailable
}
hasher := m.Hash.New()
hasher.Write([]byte(signingString))
// Sign the string and return r, s
if r, s, err := ecdsa.Sign(rand.Reader, ecdsaKey, hasher.Sum(nil)); err == nil {
curveBits := ecdsaKey.Curve.Params().BitSize
if m.CurveBits != curveBits {
return "", ErrInvalidKey
}
keyBytes := curveBits / 8
if curveBits%8 > 0 {
keyBytes += 1
}
// We serialize the outputs (r and s) into big-endian byte arrays
// padded with zeros on the left to make sure the sizes work out.
// Output must be 2*keyBytes long.
out := make([]byte, 2*keyBytes)
fillBytesInt(r, out[0:keyBytes]) // r is assigned to the first half of output.
fillBytesInt(s, out[keyBytes:]) // s is assigned to the second half of output.
return EncodeSegment(out), nil
} else {
return "", err
}
}
func fillBytesInt(x *big.Int, buf []byte) []byte {
// Clear whole buffer. (This gets optimized into a memclr.)
for i := range buf {
buf[i] = 0
}
// This code is deeply inspired by go's own implementation but rewritten.
// Although this function is called bits it returns words
words := x.Bits()
// Words are uints as per the definition of bits.Word and thus there are usually (64) /8 bytes per word
bytesPerWord := bits.UintSize / 8
// If our buffer is longer than the expected number of words start mid-way
pos := len(buf) - len(words)*bytesPerWord
// Now iterate across the words (backwards)
for i := range words {
// Grab the last word (Which is the biggest number)
word := words[len(words)-1-i]
// Now for each byte in the word
// [abcd...] we want buf[0] = a, buf[1] = b ...
for j := bytesPerWord; j > 0; j-- {
d := byte(word)
// if our position is less than 0 then panic
if pos+j-1 >= 0 {
// set the value of the byte to the byte
buf[pos+j-1] = d
} else if d != 0 {
panic("math/big: buffer too small to fit value") // have to use the same panic string for complete compatibility
}
// shift the word 8 bits and reloop.
word >>= 8
}
pos += bytesPerWord
}
return buf
}

54
vendor/github.com/golang-jwt/jwt/ecdsa_go1.15.go generated vendored Normal file
View File

@@ -0,0 +1,54 @@
//+build go1.15
package jwt
import (
"crypto/ecdsa"
"crypto/rand"
)
// Implements the Sign method from SigningMethod
// For this signing method, key must be an ecdsa.PrivateKey struct
func (m *SigningMethodECDSA) Sign(signingString string, key interface{}) (string, error) {
// Get the key
var ecdsaKey *ecdsa.PrivateKey
switch k := key.(type) {
case *ecdsa.PrivateKey:
ecdsaKey = k
default:
return "", ErrInvalidKeyType
}
// Create the hasher
if !m.Hash.Available() {
return "", ErrHashUnavailable
}
hasher := m.Hash.New()
hasher.Write([]byte(signingString))
// Sign the string and return r, s
if r, s, err := ecdsa.Sign(rand.Reader, ecdsaKey, hasher.Sum(nil)); err == nil {
curveBits := ecdsaKey.Curve.Params().BitSize
if m.CurveBits != curveBits {
return "", ErrInvalidKey
}
keyBytes := curveBits / 8
if curveBits%8 > 0 {
keyBytes += 1
}
// We serialize the outputs (r and s) into big-endian byte arrays
// padded with zeros on the left to make sure the sizes work out.
// Output must be 2*keyBytes long.
out := make([]byte, 2*keyBytes)
r.FillBytes(out[0:keyBytes]) // r is assigned to the first half of output.
s.FillBytes(out[keyBytes:]) // s is assigned to the second half of output.
return EncodeSegment(out), nil
} else {
return "", err
}
}

View File

@@ -25,7 +25,9 @@ func ParseECPrivateKeyFromPEM(key []byte) (*ecdsa.PrivateKey, error) {
// Parse the key
var parsedKey interface{}
if parsedKey, err = x509.ParseECPrivateKey(block.Bytes); err != nil {
return nil, err
if parsedKey, err = x509.ParsePKCS8PrivateKey(block.Bytes); err != nil {
return nil, err
}
}
var pkey *ecdsa.PrivateKey

81
vendor/github.com/golang-jwt/jwt/ed25519.go generated vendored Normal file
View File

@@ -0,0 +1,81 @@
package jwt
import (
"errors"
"crypto/ed25519"
)
var (
ErrEd25519Verification = errors.New("ed25519: verification error")
)
// Implements the EdDSA family
// Expects ed25519.PrivateKey for signing and ed25519.PublicKey for verification
type SigningMethodEd25519 struct{}
// Specific instance for EdDSA
var (
SigningMethodEdDSA *SigningMethodEd25519
)
func init() {
SigningMethodEdDSA = &SigningMethodEd25519{}
RegisterSigningMethod(SigningMethodEdDSA.Alg(), func() SigningMethod {
return SigningMethodEdDSA
})
}
func (m *SigningMethodEd25519) Alg() string {
return "EdDSA"
}
// Implements the Verify method from SigningMethod
// For this verify method, key must be an ed25519.PublicKey
func (m *SigningMethodEd25519) Verify(signingString, signature string, key interface{}) error {
var err error
var ed25519Key ed25519.PublicKey
var ok bool
if ed25519Key, ok = key.(ed25519.PublicKey); !ok {
return ErrInvalidKeyType
}
if len(ed25519Key) != ed25519.PublicKeySize {
return ErrInvalidKey
}
// Decode the signature
var sig []byte
if sig, err = DecodeSegment(signature); err != nil {
return err
}
// Verify the signature
if !ed25519.Verify(ed25519Key, []byte(signingString), sig) {
return ErrEd25519Verification
}
return nil
}
// Implements the Sign method from SigningMethod
// For this signing method, key must be an ed25519.PrivateKey
func (m *SigningMethodEd25519) Sign(signingString string, key interface{}) (string, error) {
var ed25519Key ed25519.PrivateKey
var ok bool
if ed25519Key, ok = key.(ed25519.PrivateKey); !ok {
return "", ErrInvalidKeyType
}
// ed25519.Sign panics if private key not equal to ed25519.PrivateKeySize
// this allows to avoid recover usage
if len(ed25519Key) != ed25519.PrivateKeySize {
return "", ErrInvalidKey
}
// Sign the string and return the encoded result
sig := ed25519.Sign(ed25519Key, []byte(signingString))
return EncodeSegment(sig), nil
}

64
vendor/github.com/golang-jwt/jwt/ed25519_utils.go generated vendored Normal file
View File

@@ -0,0 +1,64 @@
package jwt
import (
"crypto"
"crypto/ed25519"
"crypto/x509"
"encoding/pem"
"errors"
)
var (
ErrNotEdPrivateKey = errors.New("Key is not a valid Ed25519 private key")
ErrNotEdPublicKey = errors.New("Key is not a valid Ed25519 public key")
)
// Parse PEM-encoded Edwards curve private key
func ParseEdPrivateKeyFromPEM(key []byte) (crypto.PrivateKey, error) {
var err error
// Parse PEM block
var block *pem.Block
if block, _ = pem.Decode(key); block == nil {
return nil, ErrKeyMustBePEMEncoded
}
// Parse the key
var parsedKey interface{}
if parsedKey, err = x509.ParsePKCS8PrivateKey(block.Bytes); err != nil {
return nil, err
}
var pkey ed25519.PrivateKey
var ok bool
if pkey, ok = parsedKey.(ed25519.PrivateKey); !ok {
return nil, ErrNotEdPrivateKey
}
return pkey, nil
}
// Parse PEM-encoded Edwards curve public key
func ParseEdPublicKeyFromPEM(key []byte) (crypto.PublicKey, error) {
var err error
// Parse PEM block
var block *pem.Block
if block, _ = pem.Decode(key); block == nil {
return nil, ErrKeyMustBePEMEncoded
}
// Parse the key
var parsedKey interface{}
if parsedKey, err = x509.ParsePKIXPublicKey(block.Bytes); err != nil {
return nil, err
}
var pkey ed25519.PublicKey
var ok bool
if pkey, ok = parsedKey.(ed25519.PublicKey); !ok {
return nil, ErrNotEdPublicKey
}
return pkey, nil
}

View File

@@ -10,37 +10,59 @@ import (
// This is the default claims type if you don't supply one
type MapClaims map[string]interface{}
// Compares the aud claim against cmp.
// VerifyAudience Compares the aud claim against cmp.
// If required is false, this method will return true if the value matches or is unset
func (m MapClaims) VerifyAudience(cmp string, req bool) bool {
aud, _ := m["aud"].(string)
var aud []string
switch v := m["aud"].(type) {
case string:
aud = append(aud, v)
case []string:
aud = v
case []interface{}:
for _, a := range v {
vs, ok := a.(string)
if !ok {
return false
}
aud = append(aud, vs)
}
}
return verifyAud(aud, cmp, req)
}
// Compares the exp claim against cmp.
// If required is false, this method will return true if the value matches or is unset
func (m MapClaims) VerifyExpiresAt(cmp int64, req bool) bool {
switch exp := m["exp"].(type) {
exp, ok := m["exp"]
if !ok {
return !req
}
switch expType := exp.(type) {
case float64:
return verifyExp(int64(exp), cmp, req)
return verifyExp(int64(expType), cmp, req)
case json.Number:
v, _ := exp.Int64()
v, _ := expType.Int64()
return verifyExp(v, cmp, req)
}
return req == false
return false
}
// Compares the iat claim against cmp.
// If required is false, this method will return true if the value matches or is unset
func (m MapClaims) VerifyIssuedAt(cmp int64, req bool) bool {
switch iat := m["iat"].(type) {
iat, ok := m["iat"]
if !ok {
return !req
}
switch iatType := iat.(type) {
case float64:
return verifyIat(int64(iat), cmp, req)
return verifyIat(int64(iatType), cmp, req)
case json.Number:
v, _ := iat.Int64()
v, _ := iatType.Int64()
return verifyIat(v, cmp, req)
}
return req == false
return false
}
// Compares the iss claim against cmp.
@@ -53,14 +75,18 @@ func (m MapClaims) VerifyIssuer(cmp string, req bool) bool {
// Compares the nbf claim against cmp.
// If required is false, this method will return true if the value matches or is unset
func (m MapClaims) VerifyNotBefore(cmp int64, req bool) bool {
switch nbf := m["nbf"].(type) {
nbf, ok := m["nbf"]
if !ok {
return !req
}
switch nbfType := nbf.(type) {
case float64:
return verifyNbf(int64(nbf), cmp, req)
return verifyNbf(int64(nbfType), cmp, req)
case json.Number:
v, _ := nbf.Int64()
v, _ := nbfType.Int64()
return verifyNbf(v, cmp, req)
}
return req == false
return false
}
// Validates time based claims "exp, iat, nbf".
@@ -71,17 +97,17 @@ func (m MapClaims) Valid() error {
vErr := new(ValidationError)
now := TimeFunc().Unix()
if m.VerifyExpiresAt(now, false) == false {
if !m.VerifyExpiresAt(now, false) {
vErr.Inner = errors.New("Token is expired")
vErr.Errors |= ValidationErrorExpired
}
if m.VerifyIssuedAt(now, false) == false {
if !m.VerifyIssuedAt(now, false) {
vErr.Inner = errors.New("Token used before issued")
vErr.Errors |= ValidationErrorIssuedAt
}
if m.VerifyNotBefore(now, false) == false {
if !m.VerifyNotBefore(now, false) {
vErr.Inner = errors.New("Token is not valid yet")
vErr.Errors |= ValidationErrorNotValidYet
}

View File

@@ -12,9 +12,14 @@ import (
type SigningMethodRSAPSS struct {
*SigningMethodRSA
Options *rsa.PSSOptions
// VerifyOptions is optional. If set overrides Options for rsa.VerifyPPS.
// Used to accept tokens signed with rsa.PSSSaltLengthAuto, what doesn't follow
// https://tools.ietf.org/html/rfc7518#section-3.5 but was used previously.
// See https://github.com/dgrijalva/jwt-go/issues/285#issuecomment-437451244 for details.
VerifyOptions *rsa.PSSOptions
}
// Specific instances for RS/PS and company
// Specific instances for RS/PS and company.
var (
SigningMethodPS256 *SigningMethodRSAPSS
SigningMethodPS384 *SigningMethodRSAPSS
@@ -24,13 +29,15 @@ var (
func init() {
// PS256
SigningMethodPS256 = &SigningMethodRSAPSS{
&SigningMethodRSA{
SigningMethodRSA: &SigningMethodRSA{
Name: "PS256",
Hash: crypto.SHA256,
},
&rsa.PSSOptions{
Options: &rsa.PSSOptions{
SaltLength: rsa.PSSSaltLengthEqualsHash,
},
VerifyOptions: &rsa.PSSOptions{
SaltLength: rsa.PSSSaltLengthAuto,
Hash: crypto.SHA256,
},
}
RegisterSigningMethod(SigningMethodPS256.Alg(), func() SigningMethod {
@@ -39,13 +46,15 @@ func init() {
// PS384
SigningMethodPS384 = &SigningMethodRSAPSS{
&SigningMethodRSA{
SigningMethodRSA: &SigningMethodRSA{
Name: "PS384",
Hash: crypto.SHA384,
},
&rsa.PSSOptions{
Options: &rsa.PSSOptions{
SaltLength: rsa.PSSSaltLengthEqualsHash,
},
VerifyOptions: &rsa.PSSOptions{
SaltLength: rsa.PSSSaltLengthAuto,
Hash: crypto.SHA384,
},
}
RegisterSigningMethod(SigningMethodPS384.Alg(), func() SigningMethod {
@@ -54,13 +63,15 @@ func init() {
// PS512
SigningMethodPS512 = &SigningMethodRSAPSS{
&SigningMethodRSA{
SigningMethodRSA: &SigningMethodRSA{
Name: "PS512",
Hash: crypto.SHA512,
},
&rsa.PSSOptions{
Options: &rsa.PSSOptions{
SaltLength: rsa.PSSSaltLengthEqualsHash,
},
VerifyOptions: &rsa.PSSOptions{
SaltLength: rsa.PSSSaltLengthAuto,
Hash: crypto.SHA512,
},
}
RegisterSigningMethod(SigningMethodPS512.Alg(), func() SigningMethod {
@@ -94,7 +105,12 @@ func (m *SigningMethodRSAPSS) Verify(signingString, signature string, key interf
hasher := m.Hash.New()
hasher.Write([]byte(signingString))
return rsa.VerifyPSS(rsaKey, m.Hash, hasher.Sum(nil), sig, m.Options)
opts := m.Options
if m.VerifyOptions != nil {
opts = m.VerifyOptions
}
return rsa.VerifyPSS(rsaKey, m.Hash, hasher.Sum(nil), sig, opts)
}
// Implements the Sign method from SigningMethod

View File

@@ -8,7 +8,7 @@ import (
)
var (
ErrKeyMustBePEMEncoded = errors.New("Invalid Key: Key must be PEM encoded PKCS1 or PKCS8 private key")
ErrKeyMustBePEMEncoded = errors.New("Invalid Key: Key must be a PEM encoded PKCS1 or PKCS8 key")
ErrNotRSAPrivateKey = errors.New("Key is not a valid RSA private key")
ErrNotRSAPublicKey = errors.New("Key is not a valid RSA public key")
)

View File

@@ -65,7 +65,7 @@ func (t *Token) SignedString(key interface{}) (string, error) {
func (t *Token) SigningString() (string, error) {
var err error
parts := make([]string, 2)
for i, _ := range parts {
for i := range parts {
var jsonValue []byte
if i == 0 {
if jsonValue, err = json.Marshal(t.Header); err != nil {
@@ -95,14 +95,10 @@ func ParseWithClaims(tokenString string, claims Claims, keyFunc Keyfunc) (*Token
// Encode JWT specific base64url encoding with padding stripped
func EncodeSegment(seg []byte) string {
return strings.TrimRight(base64.URLEncoding.EncodeToString(seg), "=")
return base64.RawURLEncoding.EncodeToString(seg)
}
// Decode JWT specific base64url encoding with padding stripped
func DecodeSegment(seg string) ([]byte, error) {
if l := len(seg) % 4; l > 0 {
seg += strings.Repeat("=", 4-l)
}
return base64.URLEncoding.DecodeString(seg)
return base64.RawURLEncoding.DecodeString(seg)
}

View File

@@ -5,7 +5,7 @@ go 1.15
require (
cloud.google.com/go v0.67.0 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/dgrijalva/jwt-go v3.2.0+incompatible
github.com/golang-jwt/jwt v3.2.1+incompatible
github.com/gorilla/mux v1.6.2
github.com/gorilla/pat v0.0.0-20180118222023-199c85a7f6d1
github.com/gorilla/sessions v1.1.1
@@ -14,7 +14,6 @@ require (
github.com/markbates/going v1.0.0
github.com/mrjones/oauth v0.0.0-20180629183705-f4e24b6d100c
github.com/pkg/errors v0.9.1 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/stretchr/testify v1.4.0
golang.org/x/net v0.0.0-20200930145003-4acb6c075d10 // indirect
golang.org/x/oauth2 v0.0.0-20200902213428-5d25da1a8d43

View File

@@ -1,6 +1,4 @@
cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
cloud.google.com/go v0.30.0 h1:xKvyLgk56d0nksWq49J0UyGEeUIicTl4+UBiX1NPX9g=
cloud.google.com/go v0.30.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
cloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
cloud.google.com/go v0.38.0/go.mod h1:990N+gfupTy94rShfmMCWGDn0LpTmnzTp2qbd1dvSRU=
cloud.google.com/go v0.44.1/go.mod h1:iSa0KzasP4Uvy3f1mN/7PiObzGgflwredwwASm/v6AU=
@@ -14,8 +12,8 @@ cloud.google.com/go v0.54.0/go.mod h1:1rq2OEkV3YMf6n/9ZvGWI3GWw0VoqH/1x2nd8Is/bP
cloud.google.com/go v0.56.0/go.mod h1:jr7tqZxxKOVYizybht9+26Z/gUq7tiRzu+ACVAMbKVk=
cloud.google.com/go v0.57.0/go.mod h1:oXiQ6Rzq3RAkkY7N6t3TcE6jE+CIBBbA36lwQ1JyzZs=
cloud.google.com/go v0.62.0/go.mod h1:jmCYTdRCQuc1PHIIJ/maLInMho30T/Y0M4hTdTShOYc=
cloud.google.com/go v0.63.0/go.mod h1:GmezbQc7T2snqkEXWfZ0sy0VfkB/ivI2DdtJL2DEmlg=
cloud.google.com/go v0.65.0/go.mod h1:O5N8zS7uWy9vkA9vayVHs65eM1ubvY4h553ofrNHObY=
cloud.google.com/go v0.67.0 h1:YIkzmqUfVGiGPpT98L8sVvUIkDno6UlrDxw4NR6z5ak=
cloud.google.com/go v0.67.0/go.mod h1:YNan/mUhNZFrYUor0vqrsQ0Ffl7Xtm/ACOy/vsTS858=
cloud.google.com/go/bigquery v1.0.1/go.mod h1:i/xbL2UlR5RvWAURpBYZTtm/cXjCha9lbfbpx4poX+o=
cloud.google.com/go/bigquery v1.3.0/go.mod h1:PjpwJnslEMmckchkHFfq+HTD2DmtT67aNFKH1/VBDHE=
@@ -43,12 +41,9 @@ github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5P
github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU=
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=
github.com/damonkeys/goth v1.64.2 h1:I2cB72NAa1tJBI1TcLYn7H6hBeHpwm5y8Ew+VeSZXPg=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dgrijalva/jwt-go v3.2.0+incompatible h1:7qlOGliEKZXTDg6OTjfoBKDXWrumCAMpl/TFQ4/5kLM=
github.com/dgrijalva/jwt-go v3.2.0+incompatible/go.mod h1:E3ru+11k8xSBh+hMPgOLZmtrrCbhqsmaPHjLKYnJCaQ=
github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=
@@ -56,6 +51,8 @@ github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7
github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU=
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
github.com/golang-jwt/jwt v3.2.1+incompatible h1:73Z+4BJcrTC+KczS6WvTPvRGOp1WmfEP4Q1lOd9Z/+c=
github.com/golang-jwt/jwt v3.2.1+incompatible/go.mod h1:8pz2t5EyA70fFQQSrl6XZXzqecmYZeUEB8OUGHkxJ+I=
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
@@ -67,7 +64,6 @@ github.com/golang/mock v1.4.0/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt
github.com/golang/mock v1.4.1/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
github.com/golang/mock v1.4.3/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
github.com/golang/mock v1.4.4/go.mod h1:l3mdAwkq5BuhzHwde/uurv3sEJeZMXNpwsxVWU71h+4=
github.com/golang/protobuf v1.2.0 h1:P3YflyNX/ehuJFLhxviNdFxQPkGK5cDcApsge1SqnvM=
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
@@ -133,8 +129,6 @@ github.com/lestrrat-go/jwx v0.9.0 h1:Fnd0EWzTm0kFrBPzE/PEPp9nzllES5buMkksPMjEKpM
github.com/lestrrat-go/jwx v0.9.0/go.mod h1:iEoxlYfZjvoGpuWwxUz+eR5e6KTJGsaRcy/YNA/UnBk=
github.com/markbates/going v1.0.0 h1:DQw0ZP7NbNlFGcKbcE/IVSOAFzScxRtLpd0rLMzLhq0=
github.com/markbates/going v1.0.0/go.mod h1:I6mnB4BPnEeqo85ynXIx1ZFLLbtiLHNXVgWeFO9OGOA=
github.com/markbates/goth v1.65.0 h1:IbXpMneUhqbxgJ8JP1Ghl8ghlAaVX66jWDAapU1KxqU=
github.com/markbates/goth v1.65.0/go.mod h1:65frybxoeSCfORin51KOKqAKbIh7wREIDvdCkdWj//4=
github.com/mrjones/oauth v0.0.0-20180629183705-f4e24b6d100c h1:3wkDRdxK92dF+c1ke2dtj7ZzemFWBHB9plnJOtlwdFA=
github.com/mrjones/oauth v0.0.0-20180629183705-f4e24b6d100c/go.mod h1:skjdDftzkFALcuGzYSklqYd8gvat6F1gZJ4YPVbkZpM=
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
@@ -143,10 +137,8 @@ github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZb
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
github.com/stretchr/objx v0.1.0 h1:4G4v2dO3VZwixGIRoQ5Lfboy6nUhCyYzaqnIAPPhYs4=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.2.2 h1:bSDNvY7ZPG5RlJ8otE/7V6gMiyenm9RtJ7IUVIAoJ1w=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.4.0 h1:2E4SXV/wtOkTonXsotYi4li6zVWxYlZuYNCXe9XRJyk=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
@@ -192,7 +184,6 @@ golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzB
golang.org/x/mod v0.1.1-0.20191107180719-034126e5016b/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225 h1:kNX+jCowfMYzvlSvJu5pQWEmyWFrBXJ3PBy10xKMXK8=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
@@ -218,13 +209,10 @@ golang.org/x/net v0.0.0-20200513185701-a91f0712d120/go.mod h1:qpuaurCH72eLCgpAm/
golang.org/x/net v0.0.0-20200520182314-0ba52f642ac2/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200625001655-4c5254603344/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/net v0.0.0-20200707034311-ab3426394381/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/net v0.0.0-20200813134508-3edf25e44fcc/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/net v0.0.0-20200822124328-c89045814202/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/net v0.0.0-20200927032502-5d4f70055728/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/net v0.0.0-20200930145003-4acb6c075d10 h1:YfxMZzv3PjGonQYNUaeU2+DhAdqOxerQ30JFB6WgAXo=
golang.org/x/net v0.0.0-20200930145003-4acb6c075d10/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/oauth2 v0.0.0-20180620175406-ef147856a6dd h1:QQhib242ErYDSMitlBm8V7wYCm/1a25hV8qMadIKLPA=
golang.org/x/oauth2 v0.0.0-20180620175406-ef147856a6dd/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
@@ -232,7 +220,6 @@ golang.org/x/oauth2 v0.0.0-20191202225959-858c2ad4c8b6/go.mod h1:gOpvHmFTYa4Iltr
golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20200902213428-5d25da1a8d43 h1:ld7aEMNHoBnnDAX15v1T6z31v8HwR2A9FYOuAhWqkwc=
golang.org/x/oauth2 v0.0.0-20200902213428-5d25da1a8d43/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f h1:wMNYb4v58l5UBM7MYRLPG6ZhfOqbKu7X5eyFl8ZhKvA=
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
@@ -314,7 +301,6 @@ golang.org/x/tools v0.0.0-20200515010526-7d3b6ebf133d/go.mod h1:EkVYQZoAsY45+roY
golang.org/x/tools v0.0.0-20200618134242-20370b0cb4b2/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20200729194436-6467de6f59a7/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=
golang.org/x/tools v0.0.0-20200804011535-6c149bb5ef0d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=
golang.org/x/tools v0.0.0-20200806022845-90696ccdc692/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=
golang.org/x/tools v0.0.0-20200825202427-b303f430e36d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=
golang.org/x/tools v0.0.0-20200904185747-39188db58858/go.mod h1:Cj7w3i3Rnn0Xh82ur9kSqwfTHTeVxaDqrfMjpcNT6bE=
golang.org/x/tools v0.0.0-20200929161345-d7fc70abf50f/go.mod h1:z6u4i615ZeAfBE4XtMziQW1fSVJXACjjbWkB/mvPzlU=
@@ -375,7 +361,6 @@ google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEY
google.golang.org/genproto v0.0.0-20200618031413-b414f8b61790/go.mod h1:jDfRM7FcilCzHH/e9qn6dsT145K34l5v+OpcnNgKAAA=
google.golang.org/genproto v0.0.0-20200729003335-053ba62fc06f/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20200804131852-c06518451d9c/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20200806141610-86f49bd18e98/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20200825200019-8632dd797987/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20200904004341-0bd0a958aa1d/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20200929141702-51c3e5b607fe/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=

View File

@@ -8,6 +8,7 @@ import (
"fmt"
"io/ioutil"
"net/http"
"net/url"
"strings"
"time"
@@ -82,6 +83,22 @@ type OpenIDConfig struct {
Issuer string `json:"issuer"`
}
type RefreshTokenResponse struct {
AccessToken string `json:"access_token"`
// The OpenID spec defines the ID token as an optional response field in the
// refresh token flow. As a result, a new ID token may not be returned in a successful
// response.
// See more: https://openid.net/specs/openid-connect-core-1_0.html#RefreshingAccessToken
IdToken string `json:"id_token, omitempty"`
// The OAuth spec defines the refresh token as an optional response field in the
// refresh token flow. As a result, a new refresh token may not be returned in a successful
// response.
//See more: https://www.oauth.com/oauth2-servers/making-authenticated-requests/refreshing-an-access-token/
RefreshToken string `json:"refresh_token,omitempty"`
}
// New creates a new OpenID Connect provider, and sets up important connection details.
// You should always call `openidConnect.New` to get a new Provider. Never try to create
// one manually.
@@ -202,6 +219,49 @@ func (p *Provider) RefreshToken(refreshToken string) (*oauth2.Token, error) {
return newToken, err
}
// The ID token is a fundamental part of the OpenID connect refresh token flow but is not part of the OAuth flow.
// The existing RefreshToken function leverages the OAuth library's refresh token mechanism, ignoring the refreshed
// ID token. As a result, a new function needs to be exposed (rather than changing the existing function, for backwards
// compatibility purposes) that also returns the id_token in the OpenID refresh token flow API response
// Learn more about ID tokens: https://openid.net/specs/openid-connect-core-1_0.html#IDToken
func (p *Provider) RefreshTokenWithIDToken(refreshToken string) (*RefreshTokenResponse, error) {
urlValues := url.Values{
"grant_type": {"refresh_token"},
"refresh_token": {refreshToken},
"client_id": {p.ClientKey},
"client_secret": {p.Secret},
}
req, err := http.NewRequest("POST", p.OpenIDConfig.TokenEndpoint, strings.NewReader(urlValues.Encode()))
if err != nil {
return nil, err
}
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
resp, err := p.Client().Do(req)
if err != nil {
return nil, err
}
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("Non-200 response from RefreshToken: %d, WWW-Authenticate=%s", resp.StatusCode, resp.Header.Get("WWW-Authenticate"))
}
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
return nil, err
}
resp.Body.Close()
refreshTokenResponse := &RefreshTokenResponse{}
err = json.Unmarshal(body, refreshTokenResponse)
if err != nil {
return nil, err
}
return refreshTokenResponse, nil
}
// validate according to standard, returns expiry
// http://openid.net/specs/openid-connect-core-1_0.html#IDTokenValidation
func (p *Provider) validateClaims(claims map[string]interface{}) (time.Time, error) {

View File

@@ -11,6 +11,10 @@ go:
- 1.10.x
- 1.11.x
- 1.12.x
- 1.13.x
- 1.14.x
- 1.15.x
- 1.16.x
- tip
matrix:
allow_failures:

View File

@@ -9,6 +9,7 @@ Third-party patches are essential for keeping bluemonday secure and offering the
## Guidelines
1. Do not vendor dependencies. As a security package, were we to vendor dependencies the projects that then vendor bluemonday may not receive the latest security updates to the dependencies. By not vendoring dependencies the project that implements bluemonday will vendor the latest version of any dependent packages. Vendoring is a project problem, not a package problem. bluemonday will be tested against the latest version of dependencies periodically and during any PR/merge.
2. I do not care about spelling mistakes or whitespace and I do not believe that you should either. PRs therefore must be functional in their nature or be substantial and impactful if documentation or examples.
## Submitting an Issue

View File

@@ -25,7 +25,7 @@ build:
@go build
vet:
@go vet *.go
@go vet
lint:
@golint *.go

View File

@@ -2,7 +2,7 @@
bluemonday is a HTML sanitizer implemented in Go. It is fast and highly configurable.
bluemonday takes untrusted user generated content as an input, and will return HTML that has been sanitised against a whitelist of approved HTML elements and attributes so that you can safely include the content in your web page.
bluemonday takes untrusted user generated content as an input, and will return HTML that has been sanitised against an allowlist of approved HTML elements and attributes so that you can safely include the content in your web page.
If you accept user generated content, and your server uses Go, you **need** bluemonday.
@@ -50,15 +50,15 @@ bluemonday is heavily inspired by both the [OWASP Java HTML Sanitizer](https://c
## Technical Summary
Whitelist based, you need to either build a policy describing the HTML elements and attributes to permit (and the `regexp` patterns of attributes), or use one of the supplied policies representing good defaults.
Allowlist based, you need to either build a policy describing the HTML elements and attributes to permit (and the `regexp` patterns of attributes), or use one of the supplied policies representing good defaults.
The policy containing the whitelist is applied using a fast non-validating, forward only, token-based parser implemented in the [Go net/html library](https://godoc.org/golang.org/x/net/html) by the core Go team.
The policy containing the allowlist is applied using a fast non-validating, forward only, token-based parser implemented in the [Go net/html library](https://godoc.org/golang.org/x/net/html) by the core Go team.
We expect to be supplied with well-formatted HTML (closing elements for every applicable open element, nested correctly) and so we do not focus on repairing badly nested or incomplete HTML. We focus on simply ensuring that whatever elements do exist are described in the policy whitelist and that attributes and links are safe for use on your web page. [GIGO](http://en.wikipedia.org/wiki/Garbage_in,_garbage_out) does apply and if you feed it bad HTML bluemonday is not tasked with figuring out how to make it good again.
We expect to be supplied with well-formatted HTML (closing elements for every applicable open element, nested correctly) and so we do not focus on repairing badly nested or incomplete HTML. We focus on simply ensuring that whatever elements do exist are described in the policy allowlist and that attributes and links are safe for use on your web page. [GIGO](http://en.wikipedia.org/wiki/Garbage_in,_garbage_out) does apply and if you feed it bad HTML bluemonday is not tasked with figuring out how to make it good again.
### Supported Go Versions
bluemonday is tested against Go 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9, 1.10, 1.11, 1.12, and tip.
bluemonday is tested on all versions since Go 1.2 including tip.
We do not support Go 1.0 as we depend on `golang.org/x/net/html` which includes a reference to `io.ErrNoProgress` which did not exist in Go 1.0.
@@ -146,8 +146,8 @@ func main() {
We ship two default policies:
1. `bluemonday.StrictPolicy()` which can be thought of as equivalent to stripping all HTML elements and their attributes as it has nothing on its whitelist. An example usage scenario would be blog post titles where HTML tags are not expected at all and if they are then the elements *and* the content of the elements should be stripped. This is a *very* strict policy.
2. `bluemonday.UGCPolicy()` which allows a broad selection of HTML elements and attributes that are safe for user generated content. Note that this policy does *not* whitelist iframes, object, embed, styles, script, etc. An example usage scenario would be blog post bodies where a variety of formatting is expected along with the potential for TABLEs and IMGs.
1. `bluemonday.StrictPolicy()` which can be thought of as equivalent to stripping all HTML elements and their attributes as it has nothing on its allowlist. An example usage scenario would be blog post titles where HTML tags are not expected at all and if they are then the elements *and* the content of the elements should be stripped. This is a *very* strict policy.
2. `bluemonday.UGCPolicy()` which allows a broad selection of HTML elements and attributes that are safe for user generated content. Note that this policy does *not* allow iframes, object, embed, styles, script, etc. An example usage scenario would be blog post bodies where a variety of formatting is expected along with the potential for TABLEs and IMGs.
## Policy Building
@@ -220,7 +220,7 @@ p.AllowElements("fieldset", "select", "option")
### Inline CSS
Although it's possible to handle inline CSS using `AllowAttrs` with a `Matching` rule, writing a single monolithic regular expression to safely process all inline CSS which you wish to allow is not a trivial task. Instead of attempting to do so, you can whitelist the `style` attribute on whichever element(s) you desire and use style policies to control and sanitize inline styles.
Although it's possible to handle inline CSS using `AllowAttrs` with a `Matching` rule, writing a single monolithic regular expression to safely process all inline CSS which you wish to allow is not a trivial task. Instead of attempting to do so, you can allow the `style` attribute on whichever element(s) you desire and use style policies to control and sanitize inline styles.
It is suggested that you use `Matching` (with a suitable regular expression)
`MatchingEnum`, or `MatchingHandler` to ensure each style matches your needs,
@@ -241,7 +241,7 @@ p.AllowAttrs("style").OnElements("span", "p")
p.AllowStyles("text-decoration").MatchingEnum("underline", "line-through", "none").OnElements("span")
```
Or you can specify elements based on a regex patterm match:
Or you can specify elements based on a regex pattern match:
```go
p.AllowAttrs("style").OnElementsMatching(regex.MustCompile(`^my-element-`))
// Allow the 'text-decoration' property to be set to 'underline', 'line-through' or 'none'
@@ -254,6 +254,7 @@ validate the values for a given property. The string parameter has been
converted to lowercase and unicode code points have been converted.
```go
myHandler := func(value string) bool{
// Validate your input here
return true
}
p.AllowAttrs("style").OnElements("span", "p")
@@ -279,12 +280,12 @@ We provide some additional global options for safely working with links.
p.RequireParseableURLs(true)
```
If you have enabled parseable URLs then the following option will `AllowRelativeURLs`. By default this is disabled (bluemonday is a whitelist tool... you need to explicitly tell us to permit things) and when disabled it will prevent all local and scheme relative URLs (i.e. `href="localpage.html"`, `href="../home.html"` and even `href="//www.google.com"` are relative):
If you have enabled parseable URLs then the following option will `AllowRelativeURLs`. By default this is disabled (bluemonday is an allowlist tool... you need to explicitly tell us to permit things) and when disabled it will prevent all local and scheme relative URLs (i.e. `href="localpage.html"`, `href="../home.html"` and even `href="//www.google.com"` are relative):
```go
p.AllowRelativeURLs(true)
```
If you have enabled parseable URLs then you can whitelist the schemes (commonly called protocol when thinking of `http` and `https`) that are permitted. Bear in mind that allowing relative URLs in the above option will allow for a blank scheme:
If you have enabled parseable URLs then you can allow the schemes (commonly called protocol when thinking of `http` and `https`) that are permitted. Bear in mind that allowing relative URLs in the above option will allow for a blank scheme:
```go
p.AllowURLSchemes("mailto", "http", "https")
```
@@ -302,7 +303,7 @@ p.RequireNoReferrerOnLinks(true)
```
We provide a convenience method that applies all of the above, but you will still need to whitelist the linkable elements for the URL rules to be applied to:
We provide a convenience method that applies all of the above, but you will still need to allow the linkable elements for the URL rules to be applied to:
```go
p.AllowStandardURLs()
p.AllowAttrs("cite").OnElements("blockquote", "q")
@@ -372,11 +373,11 @@ p.AllowAttrs(
)
```
Both examples exhibit the same issue, they declare attributes but do not then specify whether they are whitelisted globally or only on specific elements (and which elements). Attributes belong to one or more elements, and the policy needs to declare this.
Both examples exhibit the same issue, they declare attributes but do not then specify whether they are allowed globally or only on specific elements (and which elements). Attributes belong to one or more elements, and the policy needs to declare this.
## Limitations
We are not yet including any tools to help whitelist and sanitize CSS. Which means that unless you wish to do the heavy lifting in a single regular expression (inadvisable), **you should not allow the "style" attribute anywhere**.
We are not yet including any tools to help allow and sanitize CSS. Which means that unless you wish to do the heavy lifting in a single regular expression (inadvisable), **you should not allow the "style" attribute anywhere**.
It is not the job of bluemonday to fix your bad HTML, it is merely the job of bluemonday to prevent malicious HTML getting through. If you have mismatched HTML elements, or non-conforming nesting of elements, those will remain. But if you have well-structured HTML bluemonday will not break it.

View File

@@ -27,7 +27,7 @@
// OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
package bluemonday
package css
import (
"regexp"
@@ -329,7 +329,7 @@ func splitValues(value string) []string {
return values
}
func getDefaultHandler(attr string) func(string) bool {
func GetDefaultHandler(attr string) func(string) bool {
if defaultStyleHandlers[attr] != nil {
return defaultStyleHandlers[attr]

View File

@@ -28,10 +28,10 @@
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
/*
Package bluemonday provides a way of describing a whitelist of HTML elements
Package bluemonday provides a way of describing an allowlist of HTML elements
and attributes as a policy, and for that policy to be applied to untrusted
strings from users that may contain markup. All elements and attributes not on
the whitelist will be stripped.
the allowlist will be stripped.
The default bluemonday.UGCPolicy().Sanitize() turns this:
@@ -84,21 +84,21 @@ bluemonday is heavily inspired by both the OWASP Java HTML Sanitizer
We ship two default policies, one is bluemonday.StrictPolicy() and can be
thought of as equivalent to stripping all HTML elements and their attributes as
it has nothing on its whitelist.
it has nothing on its allowlist.
The other is bluemonday.UGCPolicy() and allows a broad selection of HTML
elements and attributes that are safe for user generated content. Note that
this policy does not whitelist iframes, object, embed, styles, script, etc.
this policy does not allow iframes, object, embed, styles, script, etc.
The essence of building a policy is to determine which HTML elements and
attributes are considered safe for your scenario. OWASP provide an XSS
prevention cheat sheet ( https://www.google.com/search?q=xss+prevention+cheat+sheet )
to help explain the risks, but essentially:
1. Avoid whitelisting anything other than plain HTML elements
2. Avoid whitelisting `script`, `style`, `iframe`, `object`, `embed`, `base`
1. Avoid allowing anything other than plain HTML elements
2. Avoid allowing `script`, `style`, `iframe`, `object`, `embed`, `base`
elements
3. Avoid whitelisting anything other than plain HTML elements with simple
3. Avoid allowing anything other than plain HTML elements with simple
values that you can match to a regexp
*/
package bluemonday

View File

@@ -3,7 +3,8 @@ module github.com/microcosm-cc/bluemonday
go 1.16
require (
github.com/asaskevich/govalidator v0.0.0-20210307081110-f21760c49a8d
github.com/aymerick/douceur v0.2.0
github.com/gorilla/css v1.0.0 // indirect
golang.org/x/net v0.0.0-20210331212208-0fccb6fa2b5c
golang.org/x/net v0.0.0-20210614182718-04defd469f4e
)

View File

@@ -1,11 +1,18 @@
github.com/asaskevich/govalidator v0.0.0-20210307081110-f21760c49a8d h1:Byv0BzEl3/e6D5CLfI0j/7hiIEtvGVFPCZ7Ei2oq8iQ=
github.com/asaskevich/govalidator v0.0.0-20210307081110-f21760c49a8d/go.mod h1:WaHUgvxTVq04UNunO+XhnAqY/wQc+bxr74GqbsZ/Jqw=
github.com/aymerick/douceur v0.2.0 h1:Mv+mAeH1Q+n9Fr+oyamOlAkUNPWPlA8PPGR0QAaYuPk=
github.com/aymerick/douceur v0.2.0/go.mod h1:wlT5vV2O3h55X9m7iVYN0TBM0NH/MmbLnd30/FjWUq4=
github.com/gorilla/css v1.0.0 h1:BQqNyPTi50JCFMTw/b67hByjMVXZRwGha6wxVGkeihY=
github.com/gorilla/css v1.0.0/go.mod h1:Dn721qIggHpt4+EFCcTLTU/vk5ySda2ReITrtgBl60c=
golang.org/x/net v0.0.0-20210331212208-0fccb6fa2b5c h1:KHUzaHIpjWVlVVNh65G3hhuj3KB1HnjY6Cq5cTvRQT8=
golang.org/x/net v0.0.0-20210331212208-0fccb6fa2b5c/go.mod h1:p54w0d4576C0XHj96bSt6lcn1PtDYWL6XObtHCRCNQM=
golang.org/x/net v0.0.0-20210421230115-4e50805a0758 h1:aEpZnXcAmXkd6AvLb2OPt+EN1Zu/8Ne3pCqPjja5PXY=
golang.org/x/net v0.0.0-20210421230115-4e50805a0758/go.mod h1:72T/g9IO56b78aLF+1Kcs5dz7/ng1VjMUvfKvpfy+jM=
golang.org/x/net v0.0.0-20210610132358-84b48f89b13b h1:k+E048sYJHyVnsr1GDrRZWQ32D2C7lWs9JRc0bel53A=
golang.org/x/net v0.0.0-20210610132358-84b48f89b13b/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20210614182718-04defd469f4e h1:XpT3nA5TvE525Ne3hInMh6+GETgn27Zfm9dxsThnX2Q=
golang.org/x/net v0.0.0-20210614182718-04defd469f4e/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210420072515-93ed5bcd2bfe/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=

View File

@@ -141,7 +141,7 @@ func (p *Policy) AllowStandardURLs() {
}
// AllowStandardAttributes will enable "id", "title" and the language specific
// attributes "dir" and "lang" on all elements that are whitelisted
// attributes "dir" and "lang" on all elements that are allowed
func (p *Policy) AllowStandardAttributes() {
// "dir" "lang" are permitted as both language attributes affect charsets
// and direction of text.

View File

@@ -35,9 +35,11 @@ import (
"net/url"
"regexp"
"strings"
"github.com/microcosm-cc/bluemonday/css"
)
// Policy encapsulates the whitelist of HTML elements and attributes that will
// Policy encapsulates the allowlist of HTML elements and attributes that will
// be applied to the sanitised HTML.
//
// You should use bluemonday.NewPolicy() to create a blank policy as the
@@ -86,28 +88,31 @@ type Policy struct {
// When true, allow data attributes.
allowDataAttributes bool
// map[htmlElementName]map[htmlAttributeName]attrPolicy
elsAndAttrs map[string]map[string]attrPolicy
// When true, allow comments.
allowComments bool
// map[htmlElementName]map[htmlAttributeName][]attrPolicy
elsAndAttrs map[string]map[string][]attrPolicy
// elsMatchingAndAttrs stores regex based element matches along with attributes
elsMatchingAndAttrs map[*regexp.Regexp]map[string]attrPolicy
elsMatchingAndAttrs map[*regexp.Regexp]map[string][]attrPolicy
// map[htmlAttributeName]attrPolicy
globalAttrs map[string]attrPolicy
// map[htmlAttributeName][]attrPolicy
globalAttrs map[string][]attrPolicy
// map[htmlElementName]map[cssPropertyName]stylePolicy
elsAndStyles map[string]map[string]stylePolicy
// map[htmlElementName]map[cssPropertyName][]stylePolicy
elsAndStyles map[string]map[string][]stylePolicy
// map[regex]map[cssPropertyName]stylePolicy
elsMatchingAndStyles map[*regexp.Regexp]map[string]stylePolicy
// map[regex]map[cssPropertyName][]stylePolicy
elsMatchingAndStyles map[*regexp.Regexp]map[string][]stylePolicy
// map[cssPropertyName]stylePolicy
globalStyles map[string]stylePolicy
// map[cssPropertyName][]stylePolicy
globalStyles map[string][]stylePolicy
// If urlPolicy is nil, all URLs with matching schema are allowed.
// Otherwise, only the URLs with matching schema and urlPolicy(url)
// returning true are allowed.
allowURLSchemes map[string]urlPolicy
allowURLSchemes map[string][]urlPolicy
// If an element has had all attributes removed as a result of a policy
// being applied, then the element would be removed from the output.
@@ -174,22 +179,22 @@ type urlPolicy func(url *url.URL) (allowUrl bool)
// init initializes the maps if this has not been done already
func (p *Policy) init() {
if !p.initialized {
p.elsAndAttrs = make(map[string]map[string]attrPolicy)
p.elsMatchingAndAttrs = make(map[*regexp.Regexp]map[string]attrPolicy)
p.globalAttrs = make(map[string]attrPolicy)
p.elsAndStyles = make(map[string]map[string]stylePolicy)
p.elsMatchingAndStyles = make(map[*regexp.Regexp]map[string]stylePolicy)
p.globalStyles = make(map[string]stylePolicy)
p.allowURLSchemes = make(map[string]urlPolicy)
p.elsAndAttrs = make(map[string]map[string][]attrPolicy)
p.elsMatchingAndAttrs = make(map[*regexp.Regexp]map[string][]attrPolicy)
p.globalAttrs = make(map[string][]attrPolicy)
p.elsAndStyles = make(map[string]map[string][]stylePolicy)
p.elsMatchingAndStyles = make(map[*regexp.Regexp]map[string][]stylePolicy)
p.globalStyles = make(map[string][]stylePolicy)
p.allowURLSchemes = make(map[string][]urlPolicy)
p.setOfElementsAllowedWithoutAttrs = make(map[string]struct{})
p.setOfElementsToSkipContent = make(map[string]struct{})
p.initialized = true
}
}
// NewPolicy returns a blank policy with nothing whitelisted or permitted. This
// NewPolicy returns a blank policy with nothing allowed or permitted. This
// is the recommended way to start building a policy and you should now use
// AllowAttrs() and/or AllowElements() to construct the whitelist of HTML
// AllowAttrs() and/or AllowElements() to construct the allowlist of HTML
// elements and attributes.
func NewPolicy() *Policy {
@@ -203,7 +208,7 @@ func NewPolicy() *Policy {
// AllowAttrs takes a range of HTML attribute names and returns an
// attribute policy builder that allows you to specify the pattern and scope of
// the whitelisted attribute.
// the allowed attribute.
//
// The attribute policy is only added to the core policy when either Globally()
// or OnElements(...) are called.
@@ -223,7 +228,7 @@ func (p *Policy) AllowAttrs(attrNames ...string) *attrPolicyBuilder {
return &abp
}
// AllowDataAttributes whitelists all data attributes. We can't specify the name
// AllowDataAttributes permits all data attributes. We can't specify the name
// of each attribute exactly as they are customized.
//
// NOTE: These values are not sanitized and applications that evaluate or process
@@ -238,6 +243,22 @@ func (p *Policy) AllowDataAttributes() {
p.allowDataAttributes = true
}
// AllowComments allows comments.
//
// Please note that only one type of comment will be allowed by this, this is the
// the standard HTML comment <!-- --> which includes the use of that to permit
// conditionals as per https://docs.microsoft.com/en-us/previous-versions/windows/internet-explorer/ie-developer/compatibility/ms537512(v=vs.85)?redirectedfrom=MSDN
//
// What is not permitted are CDATA XML comments, as the x/net/html package we depend
// on does not handle this fully and we are not choosing to take on that work:
// https://pkg.go.dev/golang.org/x/net/html#Tokenizer.AllowCDATA . If the x/net/html
// package changes this then these will be considered, otherwise if you AllowComments
// but provide a CDATA comment, then as per the documentation in x/net/html this will
// be treated as a plain HTML comment.
func (p *Policy) AllowComments() {
p.allowComments = true
}
// AllowNoAttrs says that attributes on element are optional.
//
// The attribute policy is only added to the core policy when OnElements(...)
@@ -265,8 +286,7 @@ func (abp *attrPolicyBuilder) AllowNoAttrs() *attrPolicyBuilder {
}
// Matching allows a regular expression to be applied to a nascent attribute
// policy, and returns the attribute policy. Calling this more than once will
// replace the existing regexp.
// policy, and returns the attribute policy.
func (abp *attrPolicyBuilder) Matching(regex *regexp.Regexp) *attrPolicyBuilder {
abp.regexp = regex
@@ -284,7 +304,7 @@ func (abp *attrPolicyBuilder) OnElements(elements ...string) *Policy {
for _, attr := range abp.attrNames {
if _, ok := abp.p.elsAndAttrs[element]; !ok {
abp.p.elsAndAttrs[element] = make(map[string]attrPolicy)
abp.p.elsAndAttrs[element] = make(map[string][]attrPolicy)
}
ap := attrPolicy{}
@@ -292,14 +312,14 @@ func (abp *attrPolicyBuilder) OnElements(elements ...string) *Policy {
ap.regexp = abp.regexp
}
abp.p.elsAndAttrs[element][attr] = ap
abp.p.elsAndAttrs[element][attr] = append(abp.p.elsAndAttrs[element][attr], ap)
}
if abp.allowEmpty {
abp.p.setOfElementsAllowedWithoutAttrs[element] = struct{}{}
if _, ok := abp.p.elsAndAttrs[element]; !ok {
abp.p.elsAndAttrs[element] = make(map[string]attrPolicy)
abp.p.elsAndAttrs[element] = make(map[string][]attrPolicy)
}
}
}
@@ -312,19 +332,19 @@ func (abp *attrPolicyBuilder) OnElements(elements ...string) *Policy {
func (abp *attrPolicyBuilder) OnElementsMatching(regex *regexp.Regexp) *Policy {
for _, attr := range abp.attrNames {
if _, ok := abp.p.elsMatchingAndAttrs[regex]; !ok {
abp.p.elsMatchingAndAttrs[regex] = make(map[string]attrPolicy)
abp.p.elsMatchingAndAttrs[regex] = make(map[string][]attrPolicy)
}
ap := attrPolicy{}
if abp.regexp != nil {
ap.regexp = abp.regexp
}
abp.p.elsMatchingAndAttrs[regex][attr] = ap
abp.p.elsMatchingAndAttrs[regex][attr] = append(abp.p.elsMatchingAndAttrs[regex][attr], ap)
}
if abp.allowEmpty {
abp.p.setOfElementsMatchingAllowedWithoutAttrs = append(abp.p.setOfElementsMatchingAllowedWithoutAttrs, regex)
if _, ok := abp.p.elsMatchingAndAttrs[regex]; !ok {
abp.p.elsMatchingAndAttrs[regex] = make(map[string]attrPolicy)
abp.p.elsMatchingAndAttrs[regex] = make(map[string][]attrPolicy)
}
}
@@ -337,7 +357,7 @@ func (abp *attrPolicyBuilder) Globally() *Policy {
for _, attr := range abp.attrNames {
if _, ok := abp.p.globalAttrs[attr]; !ok {
abp.p.globalAttrs[attr] = attrPolicy{}
abp.p.globalAttrs[attr] = []attrPolicy{}
}
ap := attrPolicy{}
@@ -345,7 +365,7 @@ func (abp *attrPolicyBuilder) Globally() *Policy {
ap.regexp = abp.regexp
}
abp.p.globalAttrs[attr] = ap
abp.p.globalAttrs[attr] = append(abp.p.globalAttrs[attr], ap)
}
return abp.p
@@ -353,7 +373,7 @@ func (abp *attrPolicyBuilder) Globally() *Policy {
// AllowStyles takes a range of CSS property names and returns a
// style policy builder that allows you to specify the pattern and scope of
// the whitelisted property.
// the allowed property.
//
// The style policy is only added to the core policy when either Globally()
// or OnElements(...) are called.
@@ -373,8 +393,7 @@ func (p *Policy) AllowStyles(propertyNames ...string) *stylePolicyBuilder {
}
// Matching allows a regular expression to be applied to a nascent style
// policy, and returns the style policy. Calling this more than once will
// replace the existing regexp.
// policy, and returns the style policy.
func (spb *stylePolicyBuilder) Matching(regex *regexp.Regexp) *stylePolicyBuilder {
spb.regexp = regex
@@ -383,8 +402,7 @@ func (spb *stylePolicyBuilder) Matching(regex *regexp.Regexp) *stylePolicyBuilde
}
// MatchingEnum allows a list of allowed values to be applied to a nascent style
// policy, and returns the style policy. Calling this more than once will
// replace the existing list of allowed values.
// policy, and returns the style policy.
func (spb *stylePolicyBuilder) MatchingEnum(enum ...string) *stylePolicyBuilder {
spb.enum = enum
@@ -393,8 +411,7 @@ func (spb *stylePolicyBuilder) MatchingEnum(enum ...string) *stylePolicyBuilder
}
// MatchingHandler allows a handler to be applied to a nascent style
// policy, and returns the style policy. Calling this more than once will
// replace the existing handler.
// policy, and returns the style policy.
func (spb *stylePolicyBuilder) MatchingHandler(handler func(string) bool) *stylePolicyBuilder {
spb.handler = handler
@@ -412,7 +429,7 @@ func (spb *stylePolicyBuilder) OnElements(elements ...string) *Policy {
for _, attr := range spb.propertyNames {
if _, ok := spb.p.elsAndStyles[element]; !ok {
spb.p.elsAndStyles[element] = make(map[string]stylePolicy)
spb.p.elsAndStyles[element] = make(map[string][]stylePolicy)
}
sp := stylePolicy{}
@@ -423,9 +440,9 @@ func (spb *stylePolicyBuilder) OnElements(elements ...string) *Policy {
} else if spb.regexp != nil {
sp.regexp = spb.regexp
} else {
sp.handler = getDefaultHandler(attr)
sp.handler = css.GetDefaultHandler(attr)
}
spb.p.elsAndStyles[element][attr] = sp
spb.p.elsAndStyles[element][attr] = append(spb.p.elsAndStyles[element][attr], sp)
}
}
@@ -439,7 +456,7 @@ func (spb *stylePolicyBuilder) OnElementsMatching(regex *regexp.Regexp) *Policy
for _, attr := range spb.propertyNames {
if _, ok := spb.p.elsMatchingAndStyles[regex]; !ok {
spb.p.elsMatchingAndStyles[regex] = make(map[string]stylePolicy)
spb.p.elsMatchingAndStyles[regex] = make(map[string][]stylePolicy)
}
sp := stylePolicy{}
@@ -450,9 +467,9 @@ func (spb *stylePolicyBuilder) OnElementsMatching(regex *regexp.Regexp) *Policy
} else if spb.regexp != nil {
sp.regexp = spb.regexp
} else {
sp.handler = getDefaultHandler(attr)
sp.handler = css.GetDefaultHandler(attr)
}
spb.p.elsMatchingAndStyles[regex][attr] = sp
spb.p.elsMatchingAndStyles[regex][attr] = append(spb.p.elsMatchingAndStyles[regex][attr], sp)
}
return spb.p
@@ -464,7 +481,7 @@ func (spb *stylePolicyBuilder) Globally() *Policy {
for _, attr := range spb.propertyNames {
if _, ok := spb.p.globalStyles[attr]; !ok {
spb.p.globalStyles[attr] = stylePolicy{}
spb.p.globalStyles[attr] = []stylePolicy{}
}
// Use only one strategy for validating styles, fallback to default
@@ -476,15 +493,15 @@ func (spb *stylePolicyBuilder) Globally() *Policy {
} else if spb.regexp != nil {
sp.regexp = spb.regexp
} else {
sp.handler = getDefaultHandler(attr)
sp.handler = css.GetDefaultHandler(attr)
}
spb.p.globalStyles[attr] = sp
spb.p.globalStyles[attr] = append(spb.p.globalStyles[attr], sp)
}
return spb.p
}
// AllowElements will append HTML elements to the whitelist without applying an
// AllowElements will append HTML elements to the allowlist without applying an
// attribute policy to those elements (the elements are permitted
// sans-attributes)
func (p *Policy) AllowElements(names ...string) *Policy {
@@ -494,17 +511,19 @@ func (p *Policy) AllowElements(names ...string) *Policy {
element = strings.ToLower(element)
if _, ok := p.elsAndAttrs[element]; !ok {
p.elsAndAttrs[element] = make(map[string]attrPolicy)
p.elsAndAttrs[element] = make(map[string][]attrPolicy)
}
}
return p
}
// AllowElementsMatching will append HTML elements to the allowlist if they
// match a regexp.
func (p *Policy) AllowElementsMatching(regex *regexp.Regexp) *Policy {
p.init()
if _, ok := p.elsMatchingAndAttrs[regex]; !ok {
p.elsMatchingAndAttrs[regex] = make(map[string]attrPolicy)
p.elsMatchingAndAttrs[regex] = make(map[string][]attrPolicy)
}
return p
}
@@ -611,7 +630,7 @@ func (p *Policy) AllowRelativeURLs(require bool) *Policy {
return p
}
// AllowURLSchemes will append URL schemes to the whitelist
// AllowURLSchemes will append URL schemes to the allowlist
// Example: p.AllowURLSchemes("mailto", "http", "https")
func (p *Policy) AllowURLSchemes(schemes ...string) *Policy {
p.init()
@@ -629,7 +648,7 @@ func (p *Policy) AllowURLSchemes(schemes ...string) *Policy {
}
// AllowURLSchemeWithCustomPolicy will append URL schemes with
// a custom URL policy to the whitelist.
// a custom URL policy to the allowlist.
// Only the URLs with matching schema and urlPolicy(url)
// returning true will be allowed.
func (p *Policy) AllowURLSchemeWithCustomPolicy(
@@ -643,13 +662,13 @@ func (p *Policy) AllowURLSchemeWithCustomPolicy(
scheme = strings.ToLower(scheme)
p.allowURLSchemes[scheme] = urlPolicy
p.allowURLSchemes[scheme] = append(p.allowURLSchemes[scheme], urlPolicy)
return p
}
// AddSpaceWhenStrippingTag states whether to add a single space " " when
// removing tags that are not whitelisted by the policy.
// removing tags that are not allowed by the policy.
//
// This is useful if you expect to strip tags in dense markup and may lose the
// value of whitespace.

View File

@@ -31,6 +31,7 @@ package bluemonday
import (
"bytes"
"fmt"
"io"
"net/url"
"regexp"
@@ -47,10 +48,11 @@ var (
dataAttributeXMLPrefix = regexp.MustCompile("^xml.+")
dataAttributeInvalidChars = regexp.MustCompile("[A-Z;]+")
cssUnicodeChar = regexp.MustCompile(`\\[0-9a-f]{1,6} ?`)
dataURIbase64Prefix = regexp.MustCompile(`^data:[^,]*;base64,`)
)
// Sanitize takes a string that contains a HTML fragment or document and applies
// the given policy whitelist.
// the given policy allowlist.
//
// It returns a HTML string that has been sanitized by the policy or an empty
// string if an error has occurred (most likely as a consequence of extremely
@@ -60,11 +62,11 @@ func (p *Policy) Sanitize(s string) string {
return s
}
return p.sanitize(strings.NewReader(s)).String()
return p.sanitizeWithBuff(strings.NewReader(s)).String()
}
// SanitizeBytes takes a []byte that contains a HTML fragment or document and applies
// the given policy whitelist.
// the given policy allowlist.
//
// It returns a []byte containing the HTML that has been sanitized by the policy
// or an empty []byte if an error has occurred (most likely as a consequence of
@@ -74,26 +76,32 @@ func (p *Policy) SanitizeBytes(b []byte) []byte {
return b
}
return p.sanitize(bytes.NewReader(b)).Bytes()
return p.sanitizeWithBuff(bytes.NewReader(b)).Bytes()
}
// SanitizeReader takes an io.Reader that contains a HTML fragment or document
// and applies the given policy whitelist.
// and applies the given policy allowlist.
//
// It returns a bytes.Buffer containing the HTML that has been sanitized by the
// policy. Errors during sanitization will merely return an empty result.
func (p *Policy) SanitizeReader(r io.Reader) *bytes.Buffer {
return p.sanitize(r)
return p.sanitizeWithBuff(r)
}
// SanitizeReaderToWriter takes an io.Reader that contains a HTML fragment or document
// and applies the given policy allowlist and writes to the provided writer returning
// an error if there is one.
func (p *Policy) SanitizeReaderToWriter(r io.Reader, w io.Writer) error {
return p.sanitize(r, w)
}
const escapedURLChars = "'<>\"\r"
func escapeUrlComponent(val string) string {
w := bytes.NewBufferString("")
func escapeUrlComponent(w stringWriterWriter, val string) error {
i := strings.IndexAny(val, escapedURLChars)
for i != -1 {
if _, err := w.WriteString(val[:i]); err != nil {
return w.String()
return err
}
var esc string
switch val[i] {
@@ -114,15 +122,15 @@ func escapeUrlComponent(val string) string {
}
val = val[i+1:]
if _, err := w.WriteString(esc); err != nil {
return w.String()
return err
}
i = strings.IndexAny(val, escapedURLChars)
}
w.WriteString(val)
return w.String()
_, err := w.WriteString(val)
return err
}
// Query represents a query
// Query represents a single part of the query string, a query param
type Query struct {
Key string
Value string
@@ -130,6 +138,10 @@ type Query struct {
}
func parseQuery(query string) (values []Query, err error) {
// This is essentially a copy of parseQuery from
// https://golang.org/src/net/url/url.go but adjusted to build our values
// based on our type, which we need to preserve the ordering of the query
// string
for query != "" {
key := query
if i := strings.IndexAny(key, "&;"); i >= 0 {
@@ -170,18 +182,18 @@ func parseQuery(query string) (values []Query, err error) {
}
func encodeQueries(queries []Query) string {
var b strings.Builder
var buff bytes.Buffer
for i, query := range queries {
b.WriteString(url.QueryEscape(query.Key))
buff.WriteString(url.QueryEscape(query.Key))
if query.HasValue {
b.WriteString("=")
b.WriteString(url.QueryEscape(query.Value))
buff.WriteString("=")
buff.WriteString(url.QueryEscape(query.Value))
}
if i < len(queries)-1 {
b.WriteString("&")
buff.WriteString("&")
}
}
return b.String()
return buff.String()
}
func sanitizedURL(val string) (string, error) {
@@ -205,45 +217,24 @@ func sanitizedURL(val string) (string, error) {
return u.String(), nil
}
func (p *Policy) writeLinkableBuf(buff *bytes.Buffer, token *html.Token) {
// do not escape multiple query parameters
tokenBuff := bytes.NewBufferString("")
tokenBuff.WriteString("<")
tokenBuff.WriteString(token.Data)
for _, attr := range token.Attr {
tokenBuff.WriteByte(' ')
tokenBuff.WriteString(attr.Key)
tokenBuff.WriteString(`="`)
switch attr.Key {
case "href", "src":
u, ok := p.validURL(attr.Val)
if !ok {
tokenBuff.WriteString(html.EscapeString(attr.Val))
continue
}
u, err := sanitizedURL(u)
if err == nil {
tokenBuff.WriteString(u)
} else {
// fallthrough
tokenBuff.WriteString(html.EscapeString(attr.Val))
}
default:
// re-apply
tokenBuff.WriteString(html.EscapeString(attr.Val))
}
tokenBuff.WriteByte('"')
// Performs the actual sanitization process.
func (p *Policy) sanitizeWithBuff(r io.Reader) *bytes.Buffer {
var buff bytes.Buffer
if err := p.sanitize(r, &buff); err != nil {
return &bytes.Buffer{}
}
if token.Type == html.SelfClosingTagToken {
tokenBuff.WriteString("/")
}
tokenBuff.WriteString(">")
buff.WriteString(tokenBuff.String())
return &buff
}
// Performs the actual sanitization process.
func (p *Policy) sanitize(r io.Reader) *bytes.Buffer {
type asStringWriter struct {
io.Writer
}
func (a *asStringWriter) WriteString(s string) (int, error) {
return a.Write([]byte(s))
}
func (p *Policy) sanitize(r io.Reader, w io.Writer) error {
// It is possible that the developer has created the policy via:
// p := bluemonday.Policy{}
// rather than:
@@ -252,8 +243,12 @@ func (p *Policy) sanitize(r io.Reader) *bytes.Buffer {
// would initiliaze the maps, then we need to do that.
p.init()
buff, ok := w.(stringWriterWriter)
if !ok {
buff = &asStringWriter{w}
}
var (
buff bytes.Buffer
skipElementContent bool
skippingElementsCount int64
skipClosingTag bool
@@ -267,11 +262,11 @@ func (p *Policy) sanitize(r io.Reader) *bytes.Buffer {
err := tokenizer.Err()
if err == io.EOF {
// End of input means end of processing
return &buff
return nil
}
// Raw tokenizer error
return &bytes.Buffer{}
return err
}
token := tokenizer.Token()
@@ -289,6 +284,10 @@ func (p *Policy) sanitize(r io.Reader) *bytes.Buffer {
case html.CommentToken:
// Comments are ignored by default
if p.allowComments {
// But if allowed then write the comment out as-is
buff.WriteString(token.String())
}
case html.StartTagToken:
@@ -303,14 +302,18 @@ func (p *Policy) sanitize(r io.Reader) *bytes.Buffer {
skippingElementsCount++
}
if p.addSpaces {
buff.WriteString(" ")
if _, err := buff.WriteString(" "); err != nil {
return err
}
}
break
}
aps = aa
}
if len(token.Attr) != 0 {
token.Attr = p.sanitizeAttrs(token.Data, token.Attr, aps)
token.Attr = escapeAttributes(
p.sanitizeAttrs(token.Data, token.Attr, aps),
)
}
if len(token.Attr) == 0 {
@@ -318,18 +321,17 @@ func (p *Policy) sanitize(r io.Reader) *bytes.Buffer {
skipClosingTag = true
closingTagToSkipStack = append(closingTagToSkipStack, token.Data)
if p.addSpaces {
buff.WriteString(" ")
if _, err := buff.WriteString(" "); err != nil {
return err
}
}
break
}
}
if !skipElementContent {
// do not escape multiple query parameters
if linkable(token.Data) {
p.writeLinkableBuf(&buff, &token)
} else {
buff.WriteString(token.String())
if _, err := buff.WriteString(token.String()); err != nil {
return err
}
}
@@ -345,7 +347,9 @@ func (p *Policy) sanitize(r io.Reader) *bytes.Buffer {
skipClosingTag = false
}
if p.addSpaces {
buff.WriteString(" ")
if _, err := buff.WriteString(" "); err != nil {
return err
}
}
break
}
@@ -366,14 +370,18 @@ func (p *Policy) sanitize(r io.Reader) *bytes.Buffer {
}
if !match {
if p.addSpaces {
buff.WriteString(" ")
if _, err := buff.WriteString(" "); err != nil {
return err
}
}
break
}
}
if !skipElementContent {
buff.WriteString(token.String())
if _, err := buff.WriteString(token.String()); err != nil {
return err
}
}
case html.SelfClosingTagToken:
@@ -383,7 +391,9 @@ func (p *Policy) sanitize(r io.Reader) *bytes.Buffer {
aa, matched := p.matchRegex(token.Data)
if !matched {
if p.addSpaces && !matched {
buff.WriteString(" ")
if _, err := buff.WriteString(" "); err != nil {
return err
}
}
break
}
@@ -391,21 +401,20 @@ func (p *Policy) sanitize(r io.Reader) *bytes.Buffer {
}
if len(token.Attr) != 0 {
token.Attr = p.sanitizeAttrs(token.Data, token.Attr, aps)
token.Attr = escapeAttributes(p.sanitizeAttrs(token.Data, token.Attr, aps))
}
if len(token.Attr) == 0 && !p.allowNoAttrs(token.Data) {
if p.addSpaces {
buff.WriteString(" ")
if _, err := buff.WriteString(" "); err != nil {
return err
}
break
}
}
if !skipElementContent {
// do not escape multiple query parameters
if linkable(token.Data) {
p.writeLinkableBuf(&buff, &token)
} else {
buff.WriteString(token.String())
if _, err := buff.WriteString(token.String()); err != nil {
return err
}
}
@@ -416,20 +425,26 @@ func (p *Policy) sanitize(r io.Reader) *bytes.Buffer {
case `script`:
// not encouraged, but if a policy allows JavaScript we
// should not HTML escape it as that would break the output
buff.WriteString(token.Data)
case `style`:
if _, err := buff.WriteString(token.Data); err != nil {
return err
}
case "style":
// not encouraged, but if a policy allows CSS styles we
// should not HTML escape it as that would break the output
buff.WriteString(token.Data)
if _, err := buff.WriteString(token.Data); err != nil {
return err
}
default:
// HTML escape the text
buff.WriteString(token.String())
if _, err := buff.WriteString(token.String()); err != nil {
return err
}
}
}
default:
// A token that didn't exist in the html package when we wrote this
return &bytes.Buffer{}
return fmt.Errorf("unknown token: %v", token)
}
}
}
@@ -440,7 +455,7 @@ func (p *Policy) sanitize(r io.Reader) *bytes.Buffer {
func (p *Policy) sanitizeAttrs(
elementName string,
attrs []html.Attribute,
aps map[string]attrPolicy,
aps map[string][]attrPolicy,
) []html.Attribute {
if len(attrs) == 0 {
@@ -465,8 +480,9 @@ func (p *Policy) sanitizeAttrs(
}
// Builds a new attribute slice based on the whether the attribute has been
// whitelisted explicitly or globally.
// allowed explicitly or globally.
cleanAttrs := []html.Attribute{}
attrsLoop:
for _, htmlAttr := range attrs {
if p.allowDataAttributes {
// If we see a data attribute, let it through.
@@ -489,27 +505,32 @@ func (p *Policy) sanitizeAttrs(
}
// Is there an element specific attribute policy that applies?
if ap, ok := aps[htmlAttr.Key]; ok {
if ap.regexp != nil {
if ap.regexp.MatchString(htmlAttr.Val) {
if apl, ok := aps[htmlAttr.Key]; ok {
for _, ap := range apl {
if ap.regexp != nil {
if ap.regexp.MatchString(htmlAttr.Val) {
cleanAttrs = append(cleanAttrs, htmlAttr)
continue attrsLoop
}
} else {
cleanAttrs = append(cleanAttrs, htmlAttr)
continue
continue attrsLoop
}
} else {
cleanAttrs = append(cleanAttrs, htmlAttr)
continue
}
}
// Is there a global attribute policy that applies?
if ap, ok := p.globalAttrs[htmlAttr.Key]; ok {
if ap.regexp != nil {
if ap.regexp.MatchString(htmlAttr.Val) {
if apl, ok := p.globalAttrs[htmlAttr.Key]; ok {
for _, ap := range apl {
if ap.regexp != nil {
if ap.regexp.MatchString(htmlAttr.Val) {
htmlAttr.Val = escapeAttribute(htmlAttr.Val)
cleanAttrs = append(cleanAttrs, htmlAttr)
}
} else {
htmlAttr.Val = escapeAttribute(htmlAttr.Val)
cleanAttrs = append(cleanAttrs, htmlAttr)
}
} else {
cleanAttrs = append(cleanAttrs, htmlAttr)
}
}
}
@@ -533,7 +554,7 @@ func (p *Policy) sanitizeAttrs(
tmpAttrs := []html.Attribute{}
for _, htmlAttr := range cleanAttrs {
switch elementName {
case "a", "area", "link":
case "a", "area", "base", "link":
if htmlAttr.Key == "href" {
if u, ok := p.validURL(htmlAttr.Val); ok {
htmlAttr.Val = u
@@ -542,7 +563,7 @@ func (p *Policy) sanitizeAttrs(
break
}
tmpAttrs = append(tmpAttrs, htmlAttr)
case "blockquote", "q":
case "blockquote", "del", "ins", "q":
if htmlAttr.Key == "cite" {
if u, ok := p.validURL(htmlAttr.Val); ok {
htmlAttr.Val = u
@@ -551,7 +572,7 @@ func (p *Policy) sanitizeAttrs(
break
}
tmpAttrs = append(tmpAttrs, htmlAttr)
case "img", "script":
case "audio", "embed", "iframe", "img", "script", "source", "track", "video":
if htmlAttr.Key == "src" {
if u, ok := p.validURL(htmlAttr.Val); ok {
htmlAttr.Val = u
@@ -576,7 +597,7 @@ func (p *Policy) sanitizeAttrs(
// Add rel="nofollow" if a "href" exists
switch elementName {
case "a", "area", "link":
case "a", "area", "base", "link":
var hrefFound bool
var externalLink bool
for _, htmlAttr := range cleanAttrs {
@@ -753,14 +774,14 @@ func (p *Policy) sanitizeAttrs(
func (p *Policy) sanitizeStyles(attr html.Attribute, elementName string) html.Attribute {
sps := p.elsAndStyles[elementName]
if len(sps) == 0 {
sps = map[string]stylePolicy{}
sps = map[string][]stylePolicy{}
// check for any matching elements, if we don't already have a policy found
// if multiple matches are found they will be overwritten, it's best
// to not have overlapping matchers
for regex, policies := range p.elsMatchingAndStyles {
if regex.MatchString(elementName) {
for k, v := range policies {
sps[k] = v
sps[k] = append(sps[k], v...)
}
}
}
@@ -778,46 +799,51 @@ func (p *Policy) sanitizeStyles(attr html.Attribute, elementName string) html.At
clean := []string{}
prefixes := []string{"-webkit-", "-moz-", "-ms-", "-o-", "mso-", "-xv-", "-atsc-", "-wap-", "-khtml-", "prince-", "-ah-", "-hp-", "-ro-", "-rim-", "-tc-"}
decLoop:
for _, dec := range decs {
addedProperty := false
tempProperty := strings.ToLower(dec.Property)
tempValue := removeUnicode(strings.ToLower(dec.Value))
for _, i := range prefixes {
tempProperty = strings.TrimPrefix(tempProperty, i)
}
if sp, ok := sps[tempProperty]; ok {
if sp.handler != nil {
if sp.handler(tempValue) {
clean = append(clean, dec.Property+": "+dec.Value)
addedProperty = true
if spl, ok := sps[tempProperty]; ok {
for _, sp := range spl {
if sp.handler != nil {
if sp.handler(tempValue) {
clean = append(clean, dec.Property+": "+dec.Value)
continue decLoop
}
} else if len(sp.enum) > 0 {
if stringInSlice(tempValue, sp.enum) {
clean = append(clean, dec.Property+": "+dec.Value)
continue decLoop
}
} else if sp.regexp != nil {
if sp.regexp.MatchString(tempValue) {
clean = append(clean, dec.Property+": "+dec.Value)
continue decLoop
}
}
} else if len(sp.enum) > 0 {
if stringInSlice(tempValue, sp.enum) {
clean = append(clean, dec.Property+": "+dec.Value)
addedProperty = true
}
} else if sp.regexp != nil {
if sp.regexp.MatchString(tempValue) {
clean = append(clean, dec.Property+": "+dec.Value)
addedProperty = true
}
continue
}
}
if sp, ok := p.globalStyles[tempProperty]; ok && !addedProperty {
if sp.handler != nil {
if sp.handler(tempValue) {
clean = append(clean, dec.Property+": "+dec.Value)
if spl, ok := p.globalStyles[tempProperty]; ok {
for _, sp := range spl {
if sp.handler != nil {
if sp.handler(tempValue) {
clean = append(clean, dec.Property+": "+dec.Value)
continue decLoop
}
} else if len(sp.enum) > 0 {
if stringInSlice(tempValue, sp.enum) {
clean = append(clean, dec.Property+": "+dec.Value)
continue decLoop
}
} else if sp.regexp != nil {
if sp.regexp.MatchString(tempValue) {
clean = append(clean, dec.Property+": "+dec.Value)
continue decLoop
}
}
} else if len(sp.enum) > 0 {
if stringInSlice(tempValue, sp.enum) {
clean = append(clean, dec.Property+": "+dec.Value)
}
} else if sp.regexp != nil {
if sp.regexp.MatchString(tempValue) {
clean = append(clean, dec.Property+": "+dec.Value)
}
continue
}
}
}
@@ -848,11 +874,28 @@ func (p *Policy) validURL(rawurl string) (string, bool) {
rawurl = strings.TrimSpace(rawurl)
// URLs cannot contain whitespace, unless it is a data-uri
if (strings.Contains(rawurl, " ") ||
if strings.Contains(rawurl, " ") ||
strings.Contains(rawurl, "\t") ||
strings.Contains(rawurl, "\n")) &&
!strings.HasPrefix(rawurl, `data:`) {
return "", false
strings.Contains(rawurl, "\n") {
if !strings.HasPrefix(rawurl, `data:`) {
return "", false
}
// Remove \r and \n from base64 encoded data to pass url.Parse.
matched := dataURIbase64Prefix.FindString(rawurl)
if matched != "" {
rawurl = matched + strings.Replace(
strings.Replace(
rawurl[len(matched):],
"\r",
"",
-1,
),
"\n",
"",
-1,
)
}
}
// URLs are valid if they parse
@@ -863,16 +906,21 @@ func (p *Policy) validURL(rawurl string) (string, bool) {
if u.Scheme != "" {
urlPolicy, ok := p.allowURLSchemes[u.Scheme]
urlPolicies, ok := p.allowURLSchemes[u.Scheme]
if !ok {
return "", false
}
if urlPolicy == nil || urlPolicy(u) == true {
if len(urlPolicies) == 0 {
return u.String(), true
}
for _, urlPolicy := range urlPolicies {
if urlPolicy(u) == true {
return u.String(), true
}
}
return "", false
}
@@ -890,7 +938,14 @@ func (p *Policy) validURL(rawurl string) (string, bool) {
func linkable(elementName string) bool {
switch elementName {
case "a", "area", "blockquote", "img", "link", "script":
case "a", "area", "base", "link":
// elements that allow .href
return true
case "blockquote", "del", "ins", "q":
// elements that allow .cite
return true
case "audio", "embed", "iframe", "img", "input", "script", "track", "video":
// elements that allow .src
return true
default:
return false
@@ -957,14 +1012,14 @@ func removeUnicode(value string) string {
return substitutedValue
}
func (p *Policy) matchRegex(elementName string) (map[string]attrPolicy, bool) {
aps := make(map[string]attrPolicy, 0)
func (p *Policy) matchRegex(elementName string) (map[string][]attrPolicy, bool) {
aps := make(map[string][]attrPolicy, 0)
matched := false
for regex, attrs := range p.elsMatchingAndAttrs {
if regex.MatchString(elementName) {
matched = true
for k, v := range attrs {
aps[k] = v
aps[k] = append(aps[k], v...)
}
}
}
@@ -989,3 +1044,18 @@ func normaliseElementName(str string) string {
`"`,
)
}
func escapeAttributes(attrs []html.Attribute) []html.Attribute {
escapedAttrs := []html.Attribute{}
for _, attr := range attrs {
attr.Val = escapeAttribute(attr.Val)
escapedAttrs = append(escapedAttrs, attr)
}
return escapedAttrs
}
func escapeAttribute(val string) string {
val = strings.Replace(val, string([]rune{'\u00A0'}), `&nbsp;`, -1)
val = strings.Replace(val, `"`, `&quot;`, -1)
return val
}

View File

@@ -0,0 +1,10 @@
// +build go1.12
package bluemonday
import "io"
type stringWriterWriter interface {
io.Writer
io.StringWriter
}

View File

@@ -0,0 +1,14 @@
// +build go1.1,!go1.12
package bluemonday
import "io"
type stringWriterWriter interface {
io.Writer
StringWriter
}
type StringWriter interface {
WriteString(s string) (n int, err error)
}

View File

@@ -663,6 +663,24 @@ func inHeadIM(p *parser) bool {
// Ignore the token.
return true
case a.Template:
// TODO: remove this divergence from the HTML5 spec.
//
// We don't handle all of the corner cases when mixing foreign
// content (i.e. <math> or <svg>) with <template>. Without this
// early return, we can get into an infinite loop, possibly because
// of the "TODO... further divergence" a little below.
//
// As a workaround, if we are mixing foreign content and templates,
// just ignore the rest of the HTML. Foreign content is rare and a
// relatively old HTML feature. Templates are also rare and a
// relatively new HTML feature. Their combination is very rare.
for _, e := range p.oe {
if e.Namespace != "" {
p.im = ignoreTheRemainingTokens
return true
}
}
p.addElement()
p.afe = append(p.afe, &scopeMarker)
p.framesetOK = false
@@ -683,7 +701,7 @@ func inHeadIM(p *parser) bool {
if !p.oe.contains(a.Template) {
return true
}
// TODO: remove this divergence from the HTML5 spec.
// TODO: remove this further divergence from the HTML5 spec.
//
// See https://bugs.chromium.org/p/chromium/issues/detail?id=829668
p.generateImpliedEndTags()
@@ -2127,6 +2145,10 @@ func afterAfterFramesetIM(p *parser) bool {
return true
}
func ignoreTheRemainingTokens(p *parser) bool {
return true
}
const whitespaceOrNUL = whitespace + "\x00"
// Section 12.2.6.5

View File

@@ -137,11 +137,13 @@ func trimOWS(x string) string {
// contains token amongst its comma-separated tokens, ASCII
// case-insensitively.
func headerValueContainsToken(v string, token string) bool {
v = trimOWS(v)
if comma := strings.IndexByte(v, ','); comma != -1 {
return tokenEqual(trimOWS(v[:comma]), token) || headerValueContainsToken(v[comma+1:], token)
for comma := strings.IndexByte(v, ','); comma != -1; comma = strings.IndexByte(v, ',') {
if tokenEqual(trimOWS(v[:comma]), token) {
return true
}
v = v[comma+1:]
}
return tokenEqual(v, token)
return tokenEqual(trimOWS(v), token)
}
// lowerASCII returns the ASCII lowercase version of b.

View File

@@ -67,15 +67,14 @@ func Transitional(transitional bool) Option {
// VerifyDNSLength sets whether a Profile should fail if any of the IDN parts
// are longer than allowed by the RFC.
//
// This option corresponds to the VerifyDnsLength flag in UTS #46.
func VerifyDNSLength(verify bool) Option {
return func(o *options) { o.verifyDNSLength = verify }
}
// RemoveLeadingDots removes leading label separators. Leading runes that map to
// dots, such as U+3002 IDEOGRAPHIC FULL STOP, are removed as well.
//
// This is the behavior suggested by the UTS #46 and is adopted by some
// browsers.
func RemoveLeadingDots(remove bool) Option {
return func(o *options) { o.removeLeadingDots = remove }
}
@@ -83,6 +82,8 @@ func RemoveLeadingDots(remove bool) Option {
// ValidateLabels sets whether to check the mandatory label validation criteria
// as defined in Section 5.4 of RFC 5891. This includes testing for correct use
// of hyphens ('-'), normalization, validity of runes, and the context rules.
// In particular, ValidateLabels also sets the CheckHyphens and CheckJoiners flags
// in UTS #46.
func ValidateLabels(enable bool) Option {
return func(o *options) {
// Don't override existing mappings, but set one that at least checks
@@ -91,25 +92,48 @@ func ValidateLabels(enable bool) Option {
o.mapping = normalize
}
o.trie = trie
o.validateLabels = enable
o.fromPuny = validateFromPunycode
o.checkJoiners = enable
o.checkHyphens = enable
if enable {
o.fromPuny = validateFromPunycode
} else {
o.fromPuny = nil
}
}
}
// CheckHyphens sets whether to check for correct use of hyphens ('-') in
// labels. Most web browsers do not have this option set, since labels such as
// "r3---sn-apo3qvuoxuxbt-j5pe" are in common use.
//
// This option corresponds to the CheckHyphens flag in UTS #46.
func CheckHyphens(enable bool) Option {
return func(o *options) { o.checkHyphens = enable }
}
// CheckJoiners sets whether to check the ContextJ rules as defined in Appendix
// A of RFC 5892, concerning the use of joiner runes.
//
// This option corresponds to the CheckJoiners flag in UTS #46.
func CheckJoiners(enable bool) Option {
return func(o *options) {
o.trie = trie
o.checkJoiners = enable
}
}
// StrictDomainName limits the set of permissible ASCII characters to those
// allowed in domain names as defined in RFC 1034 (A-Z, a-z, 0-9 and the
// hyphen). This is set by default for MapForLookup and ValidateForRegistration.
// hyphen). This is set by default for MapForLookup and ValidateForRegistration,
// but is only useful if ValidateLabels is set.
//
// This option is useful, for instance, for browsers that allow characters
// outside this range, for example a '_' (U+005F LOW LINE). See
// http://www.rfc-editor.org/std/std3.txt for more details This option
// corresponds to the UseSTD3ASCIIRules option in UTS #46.
// http://www.rfc-editor.org/std/std3.txt for more details.
//
// This option corresponds to the UseSTD3ASCIIRules flag in UTS #46.
func StrictDomainName(use bool) Option {
return func(o *options) {
o.trie = trie
o.useSTD3Rules = use
o.fromPuny = validateFromPunycode
}
return func(o *options) { o.useSTD3Rules = use }
}
// NOTE: the following options pull in tables. The tables should not be linked
@@ -117,6 +141,8 @@ func StrictDomainName(use bool) Option {
// BidiRule enables the Bidi rule as defined in RFC 5893. Any application
// that relies on proper validation of labels should include this rule.
//
// This option corresponds to the CheckBidi flag in UTS #46.
func BidiRule() Option {
return func(o *options) { o.bidirule = bidirule.ValidString }
}
@@ -152,7 +178,8 @@ func MapForLookup() Option {
type options struct {
transitional bool
useSTD3Rules bool
validateLabels bool
checkHyphens bool
checkJoiners bool
verifyDNSLength bool
removeLeadingDots bool
@@ -225,8 +252,11 @@ func (p *Profile) String() string {
if p.useSTD3Rules {
s += ":UseSTD3Rules"
}
if p.validateLabels {
s += ":ValidateLabels"
if p.checkHyphens {
s += ":CheckHyphens"
}
if p.checkJoiners {
s += ":CheckJoiners"
}
if p.verifyDNSLength {
s += ":VerifyDNSLength"
@@ -254,26 +284,29 @@ var (
punycode = &Profile{}
lookup = &Profile{options{
transitional: true,
useSTD3Rules: true,
validateLabels: true,
trie: trie,
fromPuny: validateFromPunycode,
mapping: validateAndMap,
bidirule: bidirule.ValidString,
transitional: true,
useSTD3Rules: true,
checkHyphens: true,
checkJoiners: true,
trie: trie,
fromPuny: validateFromPunycode,
mapping: validateAndMap,
bidirule: bidirule.ValidString,
}}
display = &Profile{options{
useSTD3Rules: true,
validateLabels: true,
trie: trie,
fromPuny: validateFromPunycode,
mapping: validateAndMap,
bidirule: bidirule.ValidString,
useSTD3Rules: true,
checkHyphens: true,
checkJoiners: true,
trie: trie,
fromPuny: validateFromPunycode,
mapping: validateAndMap,
bidirule: bidirule.ValidString,
}}
registration = &Profile{options{
useSTD3Rules: true,
validateLabels: true,
verifyDNSLength: true,
checkHyphens: true,
checkJoiners: true,
trie: trie,
fromPuny: validateFromPunycode,
mapping: validateRegistration,
@@ -340,7 +373,7 @@ func (p *Profile) process(s string, toASCII bool) (string, error) {
}
isBidi = isBidi || bidirule.DirectionString(u) != bidi.LeftToRight
labels.set(u)
if err == nil && p.validateLabels {
if err == nil && p.fromPuny != nil {
err = p.fromPuny(p, u)
}
if err == nil {
@@ -681,16 +714,18 @@ func (p *Profile) validateLabel(s string) (err error) {
}
return nil
}
if !p.validateLabels {
if p.checkHyphens {
if len(s) > 4 && s[2] == '-' && s[3] == '-' {
return &labelError{s, "V2"}
}
if s[0] == '-' || s[len(s)-1] == '-' {
return &labelError{s, "V3"}
}
}
if !p.checkJoiners {
return nil
}
trie := p.trie // p.validateLabels is only set if trie is set.
if len(s) > 4 && s[2] == '-' && s[3] == '-' {
return &labelError{s, "V2"}
}
if s[0] == '-' || s[len(s)-1] == '-' {
return &labelError{s, "V3"}
}
trie := p.trie // p.checkJoiners is only set if trie is set.
// TODO: merge the use of this in the trie.
v, sz := trie.lookupString(s)
x := info(v)

View File

@@ -66,15 +66,14 @@ func Transitional(transitional bool) Option {
// VerifyDNSLength sets whether a Profile should fail if any of the IDN parts
// are longer than allowed by the RFC.
//
// This option corresponds to the VerifyDnsLength flag in UTS #46.
func VerifyDNSLength(verify bool) Option {
return func(o *options) { o.verifyDNSLength = verify }
}
// RemoveLeadingDots removes leading label separators. Leading runes that map to
// dots, such as U+3002 IDEOGRAPHIC FULL STOP, are removed as well.
//
// This is the behavior suggested by the UTS #46 and is adopted by some
// browsers.
func RemoveLeadingDots(remove bool) Option {
return func(o *options) { o.removeLeadingDots = remove }
}
@@ -82,6 +81,8 @@ func RemoveLeadingDots(remove bool) Option {
// ValidateLabels sets whether to check the mandatory label validation criteria
// as defined in Section 5.4 of RFC 5891. This includes testing for correct use
// of hyphens ('-'), normalization, validity of runes, and the context rules.
// In particular, ValidateLabels also sets the CheckHyphens and CheckJoiners flags
// in UTS #46.
func ValidateLabels(enable bool) Option {
return func(o *options) {
// Don't override existing mappings, but set one that at least checks
@@ -90,25 +91,48 @@ func ValidateLabels(enable bool) Option {
o.mapping = normalize
}
o.trie = trie
o.validateLabels = enable
o.fromPuny = validateFromPunycode
o.checkJoiners = enable
o.checkHyphens = enable
if enable {
o.fromPuny = validateFromPunycode
} else {
o.fromPuny = nil
}
}
}
// CheckHyphens sets whether to check for correct use of hyphens ('-') in
// labels. Most web browsers do not have this option set, since labels such as
// "r3---sn-apo3qvuoxuxbt-j5pe" are in common use.
//
// This option corresponds to the CheckHyphens flag in UTS #46.
func CheckHyphens(enable bool) Option {
return func(o *options) { o.checkHyphens = enable }
}
// CheckJoiners sets whether to check the ContextJ rules as defined in Appendix
// A of RFC 5892, concerning the use of joiner runes.
//
// This option corresponds to the CheckJoiners flag in UTS #46.
func CheckJoiners(enable bool) Option {
return func(o *options) {
o.trie = trie
o.checkJoiners = enable
}
}
// StrictDomainName limits the set of permissable ASCII characters to those
// allowed in domain names as defined in RFC 1034 (A-Z, a-z, 0-9 and the
// hyphen). This is set by default for MapForLookup and ValidateForRegistration.
// hyphen). This is set by default for MapForLookup and ValidateForRegistration,
// but is only useful if ValidateLabels is set.
//
// This option is useful, for instance, for browsers that allow characters
// outside this range, for example a '_' (U+005F LOW LINE). See
// http://www.rfc-editor.org/std/std3.txt for more details This option
// corresponds to the UseSTD3ASCIIRules option in UTS #46.
// http://www.rfc-editor.org/std/std3.txt for more details.
//
// This option corresponds to the UseSTD3ASCIIRules flag in UTS #46.
func StrictDomainName(use bool) Option {
return func(o *options) {
o.trie = trie
o.useSTD3Rules = use
o.fromPuny = validateFromPunycode
}
return func(o *options) { o.useSTD3Rules = use }
}
// NOTE: the following options pull in tables. The tables should not be linked
@@ -116,6 +140,8 @@ func StrictDomainName(use bool) Option {
// BidiRule enables the Bidi rule as defined in RFC 5893. Any application
// that relies on proper validation of labels should include this rule.
//
// This option corresponds to the CheckBidi flag in UTS #46.
func BidiRule() Option {
return func(o *options) { o.bidirule = bidirule.ValidString }
}
@@ -152,7 +178,8 @@ func MapForLookup() Option {
type options struct {
transitional bool
useSTD3Rules bool
validateLabels bool
checkHyphens bool
checkJoiners bool
verifyDNSLength bool
removeLeadingDots bool
@@ -225,8 +252,11 @@ func (p *Profile) String() string {
if p.useSTD3Rules {
s += ":UseSTD3Rules"
}
if p.validateLabels {
s += ":ValidateLabels"
if p.checkHyphens {
s += ":CheckHyphens"
}
if p.checkJoiners {
s += ":CheckJoiners"
}
if p.verifyDNSLength {
s += ":VerifyDNSLength"
@@ -255,9 +285,10 @@ var (
punycode = &Profile{}
lookup = &Profile{options{
transitional: true,
useSTD3Rules: true,
validateLabels: true,
removeLeadingDots: true,
useSTD3Rules: true,
checkHyphens: true,
checkJoiners: true,
trie: trie,
fromPuny: validateFromPunycode,
mapping: validateAndMap,
@@ -265,8 +296,9 @@ var (
}}
display = &Profile{options{
useSTD3Rules: true,
validateLabels: true,
removeLeadingDots: true,
checkHyphens: true,
checkJoiners: true,
trie: trie,
fromPuny: validateFromPunycode,
mapping: validateAndMap,
@@ -274,8 +306,9 @@ var (
}}
registration = &Profile{options{
useSTD3Rules: true,
validateLabels: true,
verifyDNSLength: true,
checkHyphens: true,
checkJoiners: true,
trie: trie,
fromPuny: validateFromPunycode,
mapping: validateRegistration,
@@ -339,7 +372,7 @@ func (p *Profile) process(s string, toASCII bool) (string, error) {
continue
}
labels.set(u)
if err == nil && p.validateLabels {
if err == nil && p.fromPuny != nil {
err = p.fromPuny(p, u)
}
if err == nil {
@@ -629,16 +662,18 @@ func (p *Profile) validateLabel(s string) error {
if p.bidirule != nil && !p.bidirule(s) {
return &labelError{s, "B"}
}
if !p.validateLabels {
if p.checkHyphens {
if len(s) > 4 && s[2] == '-' && s[3] == '-' {
return &labelError{s, "V2"}
}
if s[0] == '-' || s[len(s)-1] == '-' {
return &labelError{s, "V3"}
}
}
if !p.checkJoiners {
return nil
}
trie := p.trie // p.validateLabels is only set if trie is set.
if len(s) > 4 && s[2] == '-' && s[3] == '-' {
return &labelError{s, "V2"}
}
if s[0] == '-' || s[len(s)-1] == '-' {
return &labelError{s, "V3"}
}
trie := p.trie // p.checkJoiners is only set if trie is set.
// TODO: merge the use of this in the trie.
v, sz := trie.lookupString(s)
x := info(v)

View File

@@ -2,8 +2,8 @@
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//go:build (arm || mips || mipsle || 386) && linux
// +build arm mips mipsle 386
//go:build (arm || mips || mipsle || 386 || ppc) && linux
// +build arm mips mipsle 386 ppc
// +build linux
package socket

View File

@@ -2,8 +2,8 @@
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//go:build (arm || mips || mipsle || 386) && (darwin || dragonfly || freebsd || linux || netbsd || openbsd)
// +build arm mips mipsle 386
//go:build (arm || mips || mipsle || 386 || ppc) && (darwin || dragonfly || freebsd || linux || netbsd || openbsd)
// +build arm mips mipsle 386 ppc
// +build darwin dragonfly freebsd linux netbsd openbsd
package socket

View File

@@ -7,25 +7,13 @@
package socket
import "net"
import (
"net"
"sync"
)
type mmsghdrs []mmsghdr
func (hs mmsghdrs) pack(ms []Message, parseFn func([]byte, string) (net.Addr, error), marshalFn func(net.Addr) []byte) error {
for i := range hs {
vs := make([]iovec, len(ms[i].Buffers))
var sa []byte
if parseFn != nil {
sa = make([]byte, sizeofSockaddrInet6)
}
if marshalFn != nil {
sa = marshalFn(ms[i].Addr)
}
hs[i].Hdr.pack(vs, ms[i].Buffers, ms[i].OOB, sa)
}
return nil
}
func (hs mmsghdrs) unpack(ms []Message, parseFn func([]byte, string) (net.Addr, error), hint string) error {
for i := range hs {
ms[i].N = int(hs[i].Len)
@@ -41,3 +29,86 @@ func (hs mmsghdrs) unpack(ms []Message, parseFn func([]byte, string) (net.Addr,
}
return nil
}
// mmsghdrsPacker packs Message-slices into mmsghdrs (re-)using pre-allocated buffers.
type mmsghdrsPacker struct {
// hs are the pre-allocated mmsghdrs.
hs mmsghdrs
// sockaddrs is the pre-allocated buffer for the Hdr.Name buffers.
// We use one large buffer for all messages and slice it up.
sockaddrs []byte
// vs are the pre-allocated iovecs.
// We allocate one large buffer for all messages and slice it up. This allows to reuse the buffer
// if the number of buffers per message is distributed differently between calls.
vs []iovec
}
func (p *mmsghdrsPacker) prepare(ms []Message) {
n := len(ms)
if n <= cap(p.hs) {
p.hs = p.hs[:n]
} else {
p.hs = make(mmsghdrs, n)
}
if n*sizeofSockaddrInet6 <= cap(p.sockaddrs) {
p.sockaddrs = p.sockaddrs[:n*sizeofSockaddrInet6]
} else {
p.sockaddrs = make([]byte, n*sizeofSockaddrInet6)
}
nb := 0
for _, m := range ms {
nb += len(m.Buffers)
}
if nb <= cap(p.vs) {
p.vs = p.vs[:nb]
} else {
p.vs = make([]iovec, nb)
}
}
func (p *mmsghdrsPacker) pack(ms []Message, parseFn func([]byte, string) (net.Addr, error), marshalFn func(net.Addr, []byte) int) mmsghdrs {
p.prepare(ms)
hs := p.hs
vsRest := p.vs
saRest := p.sockaddrs
for i := range hs {
nvs := len(ms[i].Buffers)
vs := vsRest[:nvs]
vsRest = vsRest[nvs:]
var sa []byte
if parseFn != nil {
sa = saRest[:sizeofSockaddrInet6]
saRest = saRest[sizeofSockaddrInet6:]
} else if marshalFn != nil {
n := marshalFn(ms[i].Addr, saRest)
if n > 0 {
sa = saRest[:n]
saRest = saRest[n:]
}
}
hs[i].Hdr.pack(vs, ms[i].Buffers, ms[i].OOB, sa)
}
return hs
}
var defaultMmsghdrsPool = mmsghdrsPool{
p: sync.Pool{
New: func() interface{} {
return new(mmsghdrsPacker)
},
},
}
type mmsghdrsPool struct {
p sync.Pool
}
func (p *mmsghdrsPool) Get() *mmsghdrsPacker {
return p.p.Get().(*mmsghdrsPacker)
}
func (p *mmsghdrsPool) Put(packer *mmsghdrsPacker) {
p.p.Put(packer)
}

View File

@@ -17,6 +17,9 @@ func (h *msghdr) pack(vs []iovec, bs [][]byte, oob []byte, sa []byte) {
if sa != nil {
h.Name = (*byte)(unsafe.Pointer(&sa[0]))
h.Namelen = uint32(len(sa))
} else {
h.Name = nil
h.Namelen = 0
}
}

View File

@@ -2,8 +2,8 @@
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//go:build (arm || mips || mipsle || 386) && linux
// +build arm mips mipsle 386
//go:build (arm || mips || mipsle || 386 || ppc) && linux
// +build arm mips mipsle 386 ppc
// +build linux
package socket

View File

@@ -17,14 +17,13 @@ func (c *Conn) recvMsgs(ms []Message, flags int) (int, error) {
for i := range ms {
ms[i].raceWrite()
}
hs := make(mmsghdrs, len(ms))
packer := defaultMmsghdrsPool.Get()
defer defaultMmsghdrsPool.Put(packer)
var parseFn func([]byte, string) (net.Addr, error)
if c.network != "tcp" {
parseFn = parseInetAddr
}
if err := hs.pack(ms, parseFn, nil); err != nil {
return 0, err
}
hs := packer.pack(ms, parseFn, nil)
var operr error
var n int
fn := func(s uintptr) bool {
@@ -50,14 +49,13 @@ func (c *Conn) sendMsgs(ms []Message, flags int) (int, error) {
for i := range ms {
ms[i].raceRead()
}
hs := make(mmsghdrs, len(ms))
var marshalFn func(net.Addr) []byte
packer := defaultMmsghdrsPool.Get()
defer defaultMmsghdrsPool.Put(packer)
var marshalFn func(net.Addr, []byte) int
if c.network != "tcp" {
marshalFn = marshalInetAddr
}
if err := hs.pack(ms, nil, marshalFn); err != nil {
return 0, err
}
hs := packer.pack(ms, nil, marshalFn)
var operr error
var n int
fn := func(s uintptr) bool {

View File

@@ -55,7 +55,9 @@ func (c *Conn) sendMsg(m *Message, flags int) error {
vs := make([]iovec, len(m.Buffers))
var sa []byte
if m.Addr != nil {
sa = marshalInetAddr(m.Addr)
var a [sizeofSockaddrInet6]byte
n := marshalInetAddr(m.Addr, a[:])
sa = a[:n]
}
h.pack(vs, m.Buffers, m.OOB, sa)
var operr error

View File

@@ -2,8 +2,8 @@
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//go:build aix || darwin || dragonfly || freebsd || linux || netbsd || openbsd || solaris
// +build aix darwin dragonfly freebsd linux netbsd openbsd solaris
//go:build aix || darwin || dragonfly || freebsd || linux || netbsd || openbsd || solaris || zos
// +build aix darwin dragonfly freebsd linux netbsd openbsd solaris zos
package socket
@@ -15,4 +15,7 @@ const (
sysAF_INET6 = unix.AF_INET6
sysSOCK_RAW = unix.SOCK_RAW
sizeofSockaddrInet4 = unix.SizeofSockaddrInet4
sizeofSockaddrInet6 = unix.SizeofSockaddrInet6
)

View File

@@ -1,18 +0,0 @@
// Copyright 2020 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//go:build zos
// +build zos
package socket
import "syscall"
const (
sysAF_UNSPEC = syscall.AF_UNSPEC
sysAF_INET = syscall.AF_INET
sysAF_INET6 = syscall.AF_INET6
sysSOCK_RAW = syscall.SOCK_RAW
)

View File

@@ -0,0 +1,10 @@
// Copyright 2021 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package socket
const (
sysRECVMMSG = 0x157
sysSENDMMSG = 0x15d
)

View File

@@ -17,35 +17,36 @@ import (
"time"
)
func marshalInetAddr(a net.Addr) []byte {
// marshalInetAddr writes a in sockaddr format into the buffer b.
// The buffer must be sufficiently large (sizeofSockaddrInet4/6).
// Returns the number of bytes written.
func marshalInetAddr(a net.Addr, b []byte) int {
switch a := a.(type) {
case *net.TCPAddr:
return marshalSockaddr(a.IP, a.Port, a.Zone)
return marshalSockaddr(a.IP, a.Port, a.Zone, b)
case *net.UDPAddr:
return marshalSockaddr(a.IP, a.Port, a.Zone)
return marshalSockaddr(a.IP, a.Port, a.Zone, b)
case *net.IPAddr:
return marshalSockaddr(a.IP, 0, a.Zone)
return marshalSockaddr(a.IP, 0, a.Zone, b)
default:
return nil
return 0
}
}
func marshalSockaddr(ip net.IP, port int, zone string) []byte {
func marshalSockaddr(ip net.IP, port int, zone string, b []byte) int {
if ip4 := ip.To4(); ip4 != nil {
b := make([]byte, sizeofSockaddrInet)
switch runtime.GOOS {
case "android", "illumos", "linux", "solaris", "windows":
NativeEndian.PutUint16(b[:2], uint16(sysAF_INET))
default:
b[0] = sizeofSockaddrInet
b[0] = sizeofSockaddrInet4
b[1] = sysAF_INET
}
binary.BigEndian.PutUint16(b[2:4], uint16(port))
copy(b[4:8], ip4)
return b
return sizeofSockaddrInet4
}
if ip6 := ip.To16(); ip6 != nil && ip.To4() == nil {
b := make([]byte, sizeofSockaddrInet6)
switch runtime.GOOS {
case "android", "illumos", "linux", "solaris", "windows":
NativeEndian.PutUint16(b[:2], uint16(sysAF_INET6))
@@ -58,9 +59,9 @@ func marshalSockaddr(ip net.IP, port int, zone string) []byte {
if zone != "" {
NativeEndian.PutUint32(b[24:28], uint32(zoneCache.index(zone)))
}
return b
return sizeofSockaddrInet6
}
return nil
return 0
}
func parseInetAddr(b []byte, network string) (net.Addr, error) {
@@ -77,7 +78,7 @@ func parseInetAddr(b []byte, network string) (net.Addr, error) {
var ip net.IP
var zone string
if af == sysAF_INET {
if len(b) < sizeofSockaddrInet {
if len(b) < sizeofSockaddrInet4 {
return nil, errors.New("short address")
}
ip = make(net.IP, net.IPv4len)

View File

@@ -15,6 +15,9 @@ const (
sysAF_INET6 = 0xa
sysSOCK_RAW = 0x3
sizeofSockaddrInet4 = 0x10
sizeofSockaddrInet6 = 0x1c
)
func marshalInetAddr(ip net.IP, port int, zone string) []byte {

Some files were not shown because too many files have changed in this diff Show More