AdeelK93 Github contribution chart
AdeelK93 Github Stats
AdeelK93 Most Used Languages

Activity

16 Sep 2022

Issue Comment

AdeelK93

How to add a vpc connector to the cloud function deployment

I have some cloudsql instances and also a memcache instance that runs, in a vpc. I already have a vpc connector created. How would like to write a small function that runs some custom logic and check the data in cache before returning data. I was thinking of decoupling it from cloud run and writing a cloud function for it. But I don't see a way to provide a vpc connector config when deploying using goblet.

Forked On 16 Sep 2022 at 11:30:23

AdeelK93

With the new goblet, this is:

"cloudrun_revision": {
        "vpcAccess": {
            "connector": "VPC_CONNECTOR_NAME",
            "egress": "PRIVATE_RANGES_ONLY"
        }
    } 

Commented On 16 Sep 2022 at 11:30:23
Issue Comment

AdeelK93

Link in tooltip

Hi @AdeelK93

I want to create a collapsibleTreeNetwork with a link in the tooltip. I have forked your repo so I could change the mouseout function for hiding the tooltip. I now have some time to actually go to the link. Though the link is not clickable. My three can be found here, and the code here.

I have read this article so I would think it would be possible in D3. Though while looking at the code i do not see a difference in approach.

Do you have any suggestions to get this to work?

Forked On 13 Sep 2022 at 03:12:44

AdeelK93

@askarlupka good catch. I know I haven't been great at maintaining this, but if you want to PR that, I can put out a release

Commented On 13 Sep 2022 at 03:12:44
Issue Comment

AdeelK93

[PubSub] Add a modifyAckDeadline background process

PubSub caps ackDeadlineSeconds at 600 seconds, which is inconvenient as it's much less than Cloud Run's limit of 3600 seconds.

But it looks like there's a snooze button: https://cloud.google.com/pubsub/docs/reference/rest/v1/projects.subscriptions/modifyAckDeadline

Proposal: perhaps the @app.topic decorator, when provided with a modifyAckDeadline argument, could create a background thread that keeps extending the deadline. Optionally capped at n number of deadline extensions.

This doesn't need to be in goblet, but would be quite convenient to have in goblet.

Forked On 26 Aug 2022 at 01:38:28

AdeelK93

Never mind - this looks like it's not possible with a push subscription, only pull 😞

Commented On 26 Aug 2022 at 01:38:28

AdeelK93

resp["results"]["images"][0]["name"]

Pushed On 23 Aug 2022 at 03:18:45

AdeelK93

Gateway timeout (#225)

  • ability to pass through deadline value in "x-google-backend" (closes #224 )

Pushed On 23 Aug 2022 at 03:09:01

AdeelK93

Update readme link

Pushed On 23 Aug 2022 at 03:09:01
Issue Comment

AdeelK93

[FEATURE REQUEST] Build/Push docker image to external registry

Description (what): I'd like to be able to build my docker image for cloud run using goblet and push it to an external artifact registry (outside of the one being deployed) and then have goblet deploy from that registry.

Motivation (why): As a developer, I'd like to keep all of my artifacts in one central registry.

How is this currently done?: Currently, if we deploy to a project, the artifact will be built in that projects artifact registry. In order to circumvent this, we need to do a docker build/push and then specify what image to use.

Impact: Requires additional steps and glue during our CI/CD process

Suggest a solution (optional): In .goblet/config.json add a field for the artifact registry to push to and deploy from

{
    "cloudrun":{
        "max-instances": "1",
        "artifact_registry":"us-east1-docker.pkg.dev/gcp_project/artifact_name/image_name",
        "cpu": "1",
        "memory": "256Mi",
        "service-account": "foo@bar.com",
        "timeout": "3600",
        "vpc-connector": "projects/foo/locations/us-east1/connectors/connector"
    }
} 

During the deploy process, have goblet build and push the image to that registry (potentially with a tag provided by the developer) and then once that image has been built, have goblet deploy to the cloud run specified in goblet deploy -p shazbot -l us-east1

Potentially related issues:

https://github.com/goblet/goblet/issues/163 - requests using the GCP client libraries which will require a separation of building and deploying.

Forked On 22 Aug 2022 at 03:22:01

AdeelK93

Sorry - this was totally my mistake, I was using a virtual environment with an outdated goblet - everything works as expected!! Thank you @anovis

Commented On 22 Aug 2022 at 03:22:01

AdeelK93

prints

Pushed On 21 Aug 2022 at 03:06:35

AdeelK93

Default cloud run command (#222)

  • default command

  • formatting

Pushed On 21 Aug 2022 at 02:36:02

AdeelK93

0.8.1

Pushed On 21 Aug 2022 at 02:36:02
Issue Comment

AdeelK93

[FEATURE REQUEST] Build/Push docker image to external registry

Description (what): I'd like to be able to build my docker image for cloud run using goblet and push it to an external artifact registry (outside of the one being deployed) and then have goblet deploy from that registry.

Motivation (why): As a developer, I'd like to keep all of my artifacts in one central registry.

How is this currently done?: Currently, if we deploy to a project, the artifact will be built in that projects artifact registry. In order to circumvent this, we need to do a docker build/push and then specify what image to use.

Impact: Requires additional steps and glue during our CI/CD process

Suggest a solution (optional): In .goblet/config.json add a field for the artifact registry to push to and deploy from

{
    "cloudrun":{
        "max-instances": "1",
        "artifact_registry":"us-east1-docker.pkg.dev/gcp_project/artifact_name/image_name",
        "cpu": "1",
        "memory": "256Mi",
        "service-account": "foo@bar.com",
        "timeout": "3600",
        "vpc-connector": "projects/foo/locations/us-east1/connectors/connector"
    }
} 

During the deploy process, have goblet build and push the image to that registry (potentially with a tag provided by the developer) and then once that image has been built, have goblet deploy to the cloud run specified in goblet deploy -p shazbot -l us-east1

Potentially related issues:

https://github.com/goblet/goblet/issues/163 - requests using the GCP client libraries which will require a separation of building and deploying.

Forked On 19 Aug 2022 at 04:28:23

AdeelK93

Thanks @anovis! I have overcome the other breaking changes. But I am not able to use the external registry the way the config suggests. I've been looking through cloudbuild API documentation and haven't quite been able to figure out why.

Regardless of what registry I pass to artifact_registry - the default one is being used. When I go to the cloudbuild UI, I can see that the Image specified is not the registry I'm providing

Commented On 19 Aug 2022 at 04:28:23

AdeelK93

formatting

Pushed On 18 Aug 2022 at 03:35:43
Pull Request

AdeelK93

Default cloud run command

Created On 18 Aug 2022 at 03:35:05

AdeelK93

default command

Pushed On 18 Aug 2022 at 03:34:34

AdeelK93

default command

Pushed On 18 Aug 2022 at 03:22:01
Issue Comment

AdeelK93

[FEATURE REQUEST] Build/Push docker image to external registry

Description (what): I'd like to be able to build my docker image for cloud run using goblet and push it to an external artifact registry (outside of the one being deployed) and then have goblet deploy from that registry.

Motivation (why): As a developer, I'd like to keep all of my artifacts in one central registry.

How is this currently done?: Currently, if we deploy to a project, the artifact will be built in that projects artifact registry. In order to circumvent this, we need to do a docker build/push and then specify what image to use.

Impact: Requires additional steps and glue during our CI/CD process

Suggest a solution (optional): In .goblet/config.json add a field for the artifact registry to push to and deploy from

{
    "cloudrun":{
        "max-instances": "1",
        "artifact_registry":"us-east1-docker.pkg.dev/gcp_project/artifact_name/image_name",
        "cpu": "1",
        "memory": "256Mi",
        "service-account": "foo@bar.com",
        "timeout": "3600",
        "vpc-connector": "projects/foo/locations/us-east1/connectors/connector"
    }
} 

During the deploy process, have goblet build and push the image to that registry (potentially with a tag provided by the developer) and then once that image has been built, have goblet deploy to the cloud run specified in goblet deploy -p shazbot -l us-east1

Potentially related issues:

https://github.com/goblet/goblet/issues/163 - requests using the GCP client libraries which will require a separation of building and deploying.

Forked On 18 Aug 2022 at 02:34:45

AdeelK93

@anovis you're right, looks like cloudbuild was failing, for reasons fixed in #220.

I'm having other issues with my config not working, which was previously working in 0.7.6, which I'll file separate PRs and issues for

Commented On 18 Aug 2022 at 02:34:45

AdeelK93

Include timezone as a schedule argument (#203)

Timezone is a pretty important part of any schedule - moving it to the function argument should make it clear how to do so.

I saw the kwargs, so I tried timeZone (what the API wants) and time_zone (what the python scheduler library wants). Had to go through the source code to find out it was timezone, so I hope this change makes things easier for other newcomers!

Pushed On 18 Aug 2022 at 02:23:16

AdeelK93

chore(Dockerfile): update python 3.7 -> 3.10 (#206)

  • Updates the base container from python:3.7-slim to python:3.10-slim
  • installs the dependencies before copying the application code to take advantage of docker build layer caching
  • updates goblet Dockerfile with the same changes

Closes https://github.com/goblet/goblet/issues/205

Pushed On 18 Aug 2022 at 02:23:16

AdeelK93

[PubSub] pass through configuration for subscriptions (#209)

  • addresses https://github.com/goblet/goblet/issues/181
  • allow pass through configuration for pubsub subscriptions
  • note: need to ensure conflicting subscription config values are not set i.e, pushConfig or enableMessageOrdering and enableExactlyOnceDelivery cannot be set at the same time

Pushed On 18 Aug 2022 at 02:23:16

AdeelK93

Split Cloud Build and Deploy and added Traffic (#208)

Fixes issues #163 and #207

Pushed On 18 Aug 2022 at 02:23:16

AdeelK93

Bugfix: Subscription attribute filter (#211)

closes #204

Pushed On 18 Aug 2022 at 02:23:16

AdeelK93

new backend classes and support for gen 2 cloudfunctions (#198)

  • supports v2

  • Update deploy.py

  • fix url problem

  • some progress, still need pubsub

  • new v2 cloudfunctions client for calling generateUploadUrl

  • source upload stuff

  • Update deploy.py

  • Update deploy.py

  • Update deploy.py

  • Update pubsub.py

  • fixing utils bug

  • add backend class and supported checks for gen 2 cloudfunctions

  • Create init.py

  • new updates

  • refactoring tests

  • tests

  • almost there, bugs in storage and eventarc

  • new changes

  • adding new cloudbuild to backends

  • rollback eventarc changes

  • all tests passing but gcf v2

  • small changes

  • cleanup tests

  • Reference to gen 2 support for eventarc

  • fix python runtime comparision

  • fix lint

  • add replay

  • Update decorators.py

  • Update decorators.py

  • Move destroy method to backend classes

  • Lint and reformatting

  • update docstring

  • fix get_cloudrun_url for v2 cloud run

  • Update common_cloud_actions.py

  • documentation

  • support for v1 cloudrun and minor fix for storage gcf v2 support

  • add v2 to supported

  • fix lint

  • fix project

  • take out support for cloudrun v1

  • getting rid of references to v2 cloudrun

  • formatting

Co-authored-by: anovis anovis@ymail.com

Pushed On 18 Aug 2022 at 02:23:16

AdeelK93

fix cloudrun delta check (#212)

  • fix cloudrun delta function

Co-authored-by: anovis anovis@ymail.com

Pushed On 18 Aug 2022 at 02:23:16

AdeelK93

Fix aliasing with rueyaml dump (#213)

  • fix aliasing with yaml

  • fix l int

Pushed On 18 Aug 2022 at 02:23:16

AdeelK93

Container Config and Project ID Fix (#216)

  • container config

  • black

Pushed On 18 Aug 2022 at 02:23:16

AdeelK93

0.8.0

Pushed On 18 Aug 2022 at 02:23:16

AdeelK93

Add waiters for api gateway destroy (#217)

  • Add waiters for api gateway destroy functions (closes #201)

Pushed On 18 Aug 2022 at 02:23:16

AdeelK93

Update readme

Pushed On 18 Aug 2022 at 02:23:16

AdeelK93

Cloudbuild Timeout (#219)

Pass through cloudbuild timeout to the cloudbuild wait_for_operation

Pushed On 18 Aug 2022 at 02:23:16

AdeelK93

Fix custom files (#220)

renamed custom_files. It now has format

"custom_files": {
      "include": ["*.yaml"],
      "exclude": ["*.secret"]
  } 

Pushed On 18 Aug 2022 at 02:23:16

AdeelK93

#119 (#221)

Update cloudrun.py

Pushed On 18 Aug 2022 at 02:23:16

AdeelK93

Merge branch 'goblet:main' into main

Pushed On 18 Aug 2022 at 02:23:16
Issue Comment

AdeelK93

[FEATURE REQUEST] Build/Push docker image to external registry

Description (what): I'd like to be able to build my docker image for cloud run using goblet and push it to an external artifact registry (outside of the one being deployed) and then have goblet deploy from that registry.

Motivation (why): As a developer, I'd like to keep all of my artifacts in one central registry.

How is this currently done?: Currently, if we deploy to a project, the artifact will be built in that projects artifact registry. In order to circumvent this, we need to do a docker build/push and then specify what image to use.

Impact: Requires additional steps and glue during our CI/CD process

Suggest a solution (optional): In .goblet/config.json add a field for the artifact registry to push to and deploy from

{
    "cloudrun":{
        "max-instances": "1",
        "artifact_registry":"us-east1-docker.pkg.dev/gcp_project/artifact_name/image_name",
        "cpu": "1",
        "memory": "256Mi",
        "service-account": "foo@bar.com",
        "timeout": "3600",
        "vpc-connector": "projects/foo/locations/us-east1/connectors/connector"
    }
} 

During the deploy process, have goblet build and push the image to that registry (potentially with a tag provided by the developer) and then once that image has been built, have goblet deploy to the cloud run specified in goblet deploy -p shazbot -l us-east1

Potentially related issues:

https://github.com/goblet/goblet/issues/163 - requests using the GCP client libraries which will require a separation of building and deploying.

Forked On 17 Aug 2022 at 08:12:09

AdeelK93

Thank you @anovis! Got cloudfunction enabled on my project, but running into another error. Does this images error look familiar to you?

INFO:goblet.app:preparing to deploy with backend CloudRun E0817 14:42:17.396142000 4590663168 fork_posix.cc:76] Other threads are currently calling into gRPC, skipping fork() handlers E0817 14:42:19.297404000 4590663168 fork_posix.cc:76] Other threads are currently calling into gRPC, skipping fork() handlers INFO:goblet.backend:zipping source code E0817 14:42:21.443381000 4590663168 fork_posix.cc:76] Other threads are currently calling into gRPC, skipping fork() handlers INFO:goblet.backend:uploading source zip to gs...... INFO:goblet.backend:source code uploaded E0817 14:42:28.906718000 4590663168 fork_posix.cc:76] Other threads are currently calling into gRPC, skipping fork() handlers INFO:goblet.deployer:creating cloudbuild Traceback (most recent call last): File "bin/goblet", line 8, in sys.exit(main()) File "python3.10/site-packages/click/core.py", line 1130, in call return self.main(*args, **kwargs) File "python3.10/site-packages/click/core.py", line 1055, in main rv = self.invoke(ctx) File "python3.10/site-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "python3.10/site-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, **ctx.params) File "python3.10/site-packages/click/core.py", line 760, in invoke return __callback(*args, **kwargs) File "python3.10/site-packages/goblet/cli.py", line 71, in deploy app.deploy(skip_function, only_function, config=config, force=False) File "python3.10/site-packages/goblet/app.py", line 65, in deploy source = self.backend_class(self).deploy(force=force, config=config) File "python3.10/site-packages/goblet/backends/cloudrun.py", line 58, in deploy serviceRevision.deployRevision() File "python3.10/site-packages/goblet/revision.py", line 121, in deployRevision self.getArtifact() File "python3.10/site-packages/goblet/revision.py", line 44, in getArtifact resp["results"]["images"][0]["name"] KeyError: 'images'

Commented On 17 Aug 2022 at 08:12:09
Issue Comment

AdeelK93

[FEATURE REQUEST] Build/Push docker image to external registry

Description (what): I'd like to be able to build my docker image for cloud run using goblet and push it to an external artifact registry (outside of the one being deployed) and then have goblet deploy from that registry.

Motivation (why): As a developer, I'd like to keep all of my artifacts in one central registry.

How is this currently done?: Currently, if we deploy to a project, the artifact will be built in that projects artifact registry. In order to circumvent this, we need to do a docker build/push and then specify what image to use.

Impact: Requires additional steps and glue during our CI/CD process

Suggest a solution (optional): In .goblet/config.json add a field for the artifact registry to push to and deploy from

{
    "cloudrun":{
        "max-instances": "1",
        "artifact_registry":"us-east1-docker.pkg.dev/gcp_project/artifact_name/image_name",
        "cpu": "1",
        "memory": "256Mi",
        "service-account": "foo@bar.com",
        "timeout": "3600",
        "vpc-connector": "projects/foo/locations/us-east1/connectors/connector"
    }
} 

During the deploy process, have goblet build and push the image to that registry (potentially with a tag provided by the developer) and then once that image has been built, have goblet deploy to the cloud run specified in goblet deploy -p shazbot -l us-east1

Potentially related issues:

https://github.com/goblet/goblet/issues/163 - requests using the GCP client libraries which will require a separation of building and deploying.

Forked On 15 Aug 2022 at 04:09:39

AdeelK93

Hi @anovis, this looks great! I'm having a bit of trouble using that config for cloud run though, it's instead trying to use cloud functions, which I do not have enabled. It shouldn't need cloud functions, should it?

googleapiclient.errors.HttpError: <HttpError 403 when requesting https://cloudfunctions.googleapis.com/v2beta/projects/PROJECT/locations/us-east1/functions:generateUploadUrl?alt=json returned "Cloud Functions API has not been used in project PROJECT before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/cloudfunctions.googleapis.com/overview?project= PROJECT then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.". Details: "[{'@type': 'type.googleapis.com/google.rpc.Help', 'links': [{'description': 'Google developers console API activation', 'url': 'https://console.developers.google.com/apis/api/cloudfunctions.googleapis.com/overview?project= PROJECT'}]}, {'@type': 'type.googleapis.com/google.rpc.ErrorInfo', 'reason': 'SERVICE_DISABLED', 'domain': 'googleapis.com', 'metadata': {'consumer': 'projects/PROJECT', 'service': 'cloudfunctions.googleapis.com'}}]">

Commented On 15 Aug 2022 at 04:09:39

AdeelK93

more testing

Pushed On 29 Jul 2022 at 05:15:43

AdeelK93

testing

Pushed On 29 Jul 2022 at 04:37:52
Merge

AdeelK93

Include timezone as a schedule argument

Timezone is a pretty important part of any schedule - moving it to the function argument should make it clear how to do so.

I saw the kwargs, so I tried timeZone (what the API wants) and time_zone (what the python scheduler library wants). Had to go through the source code to find out it was timezone, so I hope this change makes things easier for other newcomers!

Forked On 29 Jul 2022 at 04:23:45

AdeelK93

ah thanks for catching that :)
On 29 Jul 2022 at 04:23:45

AdeelK93

Include timezone as a schedule argument

Timezone is a pretty important part of any schedule - moving it to the function argument should make it clear how to do so.

I saw the kwargs, so I tried timeZone (what the API wants) and time_zone (what the python scheduler library wants). Had to go through the source code to find out it was timezone, so I hope this change makes things easier for other newcomers!

Merged On 29 Jul 2022 at 04:23:46

AdeelK93

Commented On 29 Jul 2022 at 04:23:46

AdeelK93

black

Pushed On 29 Jul 2022 at 04:22:43

AdeelK93

formatting

Pushed On 29 Jul 2022 at 04:20:49
Pull Request

AdeelK93

Include timezone as a schedule argument

Created On 29 Jul 2022 at 02:00:21

AdeelK93

explicit timezone argument for scheduler

Pushed On 29 Jul 2022 at 01:54:44

AdeelK93

Removed unused CMD (#202)

removed unused CMD

Pushed On 29 Jul 2022 at 01:43:35

AdeelK93

Merge branch 'goblet:main' into main

Pushed On 29 Jul 2022 at 01:43:35
Pull Request

AdeelK93

Removed unused CMD

Created On 28 Jul 2022 at 04:42:16

AdeelK93

removed unused CMD

Pushed On 28 Jul 2022 at 04:39:48