Hi all,
Here is what I want to do. I have a service that relies on some data. That
data is stored in a database and doesn't change much, but it has the
potential to change during the life of the service. What I would like to do
is go fetch all of this data when the service starts and store it in memory
in an easily accessible singleton that each worker instance will have
access to. If the data changes on the db I will make an rpc call to the
service to update the singleton's data.
So I've tried making a 'constants' dependency and passing my sql_alchemy
dependency to the initializer in order to make the db calls on the
'startup' call of the constants dependency. However, this doesn't work
because in order to make the get_dependency() call on the sql alchemy
dependency I need a worker_ctx. Which I don't have because this is still
start up.
My other thought was to somehow spin up a worker at startup and make a call
to this constants dependency so I could pass in the sql_alchemy dependency
but I wasn't sure how to do that and guarantee the db calls would finish
before the service started processing other requests and spinning up other
worker instances. I need this data to be populated before other requests
are processed.
Any help would be appreciated. Maybe there is a better way to do this?
Hi,
A custom dependency seems like a reasonable way to go. As you've
discovered, nameko-sqlalchemy doesn't work as a shared extension, but it's
pretty small so you can probably lift the few bits you need from the source
to put in your `start`. Doing your initialisation in your dependency
`start` definitely seems better than trying to spin up a worker
Best,
David
···
On Wednesday, 7 September 2016 17:33:11 UTC+1, dillon...@gmail.com wrote:
Hi all,
Here is what I want to do. I have a service that relies on some data. That
data is stored in a database and doesn't change much, but it has the
potential to change during the life of the service. What I would like to do
is go fetch all of this data when the service starts and store it in memory
in an easily accessible singleton that each worker instance will have
access to. If the data changes on the db I will make an rpc call to the
service to update the singleton's data.
So I've tried making a 'constants' dependency and passing my sql_alchemy
dependency to the initializer in order to make the db calls on the
'startup' call of the constants dependency. However, this doesn't work
because in order to make the get_dependency() call on the sql alchemy
dependency I need a worker_ctx. Which I don't have because this is still
start up.
My other thought was to somehow spin up a worker at startup and make a
call to this constants dependency so I could pass in the sql_alchemy
dependency but I wasn't sure how to do that and guarantee the db calls
would finish before the service started processing other requests and
spinning up other worker instances. I need this data to be populated before
other requests are processed.
Any help would be appreciated. Maybe there is a better way to do this?
Thanks for the response! That's actually exactly what I ended up doing.
Good to hear the approach 'seconded'.
···
On Wednesday, September 7, 2016 at 1:54:14 PM UTC-4, David Szotten wrote:
Hi,
A custom dependency seems like a reasonable way to go. As you've
discovered, nameko-sqlalchemy doesn't work as a shared extension, but it's
pretty small so you can probably lift the few bits you need from the source
to put in your `start`. Doing your initialisation in your dependency
`start` definitely seems better than trying to spin up a worker
Best,
David
On Wednesday, 7 September 2016 17:33:11 UTC+1, dillon...@gmail.com wrote:
Hi all,
Here is what I want to do. I have a service that relies on some data.
That data is stored in a database and doesn't change much, but it has the
potential to change during the life of the service. What I would like to do
is go fetch all of this data when the service starts and store it in memory
in an easily accessible singleton that each worker instance will have
access to. If the data changes on the db I will make an rpc call to the
service to update the singleton's data.
So I've tried making a 'constants' dependency and passing my sql_alchemy
dependency to the initializer in order to make the db calls on the
'startup' call of the constants dependency. However, this doesn't work
because in order to make the get_dependency() call on the sql alchemy
dependency I need a worker_ctx. Which I don't have because this is still
start up.
My other thought was to somehow spin up a worker at startup and make a
call to this constants dependency so I could pass in the sql_alchemy
dependency but I wasn't sure how to do that and guarantee the db calls
would finish before the service started processing other requests and
spinning up other worker instances. I need this data to be populated before
other requests are processed.
Any help would be appreciated. Maybe there is a better way to do this?