# Running tests without RabbitMQ when using worker\_factory and container\_factory

**URL:** <https://discourse.nameko.io/t/running-tests-without-rabbitmq-when-using-worker-factory-and-container-factory/138>\
**Category:** googlegroup\
**Created:** [August 10, 2016, 7:21am UTC](https://discourse.nameko.io/t/running-tests-without-rabbitmq-when-using-worker-factory-and-container-factory/138 "2016-08-10T07:21:48Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jesse\_Pollak](https://avatars.discourse-cdn.com/v4/letter/j/b5a626/32.png) [@Jesse\_Pollak](https://discourse.nameko.io/u/Jesse_Pollak)\
**Post date:** [August 10, 2016, 7:21am UTC](https://discourse.nameko.io/t/running-tests-without-rabbitmq-when-using-worker-factory-and-container-factory/138/1 "2016-08-10T07:21:48Z")

</div>

Hi! We're writing tests for our services that test their high level  
functionality using \*worker\_factory\* and \*container\_factory\*. When we use  
\*worker\_factory\*, all the service's dependencies are replaced with mock  
dependencies as specified in the docstring  
\<[https://github.com/onefinestay/nameko/blob/master/nameko/testing/services.py#L141](https://github.com/onefinestay/nameko/blob/master/nameko/testing/services.py#L141)\>.  
When we use \*container\_factory\*, we are manually mocking all services using  
the replace\_dependencies helper  
\<[https://github.com/onefinestay/nameko/blob/master/nameko/testing/services.py#L229](https://github.com/onefinestay/nameko/blob/master/nameko/testing/services.py#L229)\>,  
which replaces the dependencies with mocks. Many of these are \*RpcProxy\* dependencies  
that communicate with other services, but given the specified functionality  
of \*worker\_factory\* and \*replace\_dependencies, \*we expect these rpc-level  
dependencies to be mocked and for there to be no need for an active  
RabbitMQ cluster.

\*Unfortunately, even though all of our dependencies are mocked, when we try  
to run our tests, the tests error out on container.start() with a failed  
connection to RabbitMQ.\*

Digging into the code a bit, I believe I've found the reason, but I wanted  
to make sure I'm understanding.

I believe that while the \*dependencies\* are being replaced with mocks, \*subextensions  
\*added by those dependencies are not being replaced. The subextension  
addition logic happens in the container \*\_\_init\_\_\*  
\<[https://github.com/onefinestay/nameko/blob/master/nameko/containers.py#L160](https://github.com/onefinestay/nameko/blob/master/nameko/containers.py#L160)\>,  
which happens before both \*worker\_factory\* and \*replace\_dependencies\* replace  
the \*RpcProxy\* dependencies with mocks. Therefore, \*RpcProxy\* dependencies  
add a QueueConsumer  
\<[https://github.com/onefinestay/nameko/blob/master/nameko/rpc.py#L40](https://github.com/onefinestay/nameko/blob/master/nameko/rpc.py#L40)\> dependencies  
to \*container.subextensions\* which triggers the RabbitMQ connection.

For now, I've been able to work around this by manually overriding the  
class dependencies before I use \*container\_factory, \*like this\*:\*

ExampleService.rpc\_dependency\_1 = Mock()  
ExampleService.rpc\_dependency\_2 = Mock()  
container = container\_factory(ExampleService, web\_config)  
container.start()

However, this seems suboptimal.

Has anyone dealt with this before?

---

<div class="post-metadata">

**Author:** ![David\_Szotten](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.nameko.io/david_szotten/32/6_2.png) [@David\_Szotten](https://discourse.nameko.io/u/David_Szotten)\
**Post date:** [August 10, 2016, 8:21am UTC](https://discourse.nameko.io/t/running-tests-without-rabbitmq-when-using-worker-factory-and-container-factory/138/2 "2016-08-10T08:21:12Z")

</div>

Hi

Could you give some more detail about why/when you are using the  
container\_factory over the worker\_factory? If you want everything mocked  
out it sounds like the worker\_factory (which won't need rabbitmq) is a  
better choice. The container factory is for when you want to simulate more  
realistic conditions, and so defaults to hosting real versions of everything

If you need only a few select dependencies, say a db connection, one option  
is to still use the worker\_factory, and manually inject the db, as  
in [https://github.com/onefinestay/nameko/blob/master/docs/examples/testing/alternative\_dependency\_unit\_test.py#L52](https://github.com/onefinestay/nameko/blob/master/docs/examples/testing/alternative_dependency_unit_test.py#L52)

Best,  
David

> **···**
>
> On Wednesday, 10 August 2016 08:21:48 UTC+1, je...@pollak.io wrote:
> 
> > Hi! We're writing tests for our services that test their high level  
> > functionality using \*worker\_factory\* and \*container\_factory\*. When we use  
> > \*worker\_factory\*, all the service's dependencies are replaced with mock  
> > dependencies as specified in the docstring  
> > \<[https://github.com/onefinestay/nameko/blob/master/nameko/testing/services.py#L141&gt;\](https://github.com/onefinestay/nameko/blob/master/nameko/testing/services.py#L141&gt;%5C).  
> > When we use \*container\_factory\*, we are manually mocking all services  
> > using the replace\_dependencies helper  
> > \<[https://github.com/onefinestay/nameko/blob/master/nameko/testing/services.py#L229&gt;](https://github.com/onefinestay/nameko/blob/master/nameko/testing/services.py#L229&gt;),  
> > which replaces the dependencies with mocks. Many of these are \*RpcProxy\* dependencies  
> > that communicate with other services, but given the specified functionality  
> > of \*worker\_factory\* and \*replace\_dependencies, \*we expect these rpc-level  
> > dependencies to be mocked and for there to be no need for an active  
> > RabbitMQ cluster.
> > 
> > \*Unfortunately, even though all of our dependencies are mocked, when we  
> > try to run our tests, the tests error out on container.start() with a  
> > failed connection to RabbitMQ.\*
> > 
> > Digging into the code a bit, I believe I've found the reason, but I wanted  
> > to make sure I'm understanding.
> > 
> > I believe that while the \*dependencies\* are being replaced with mocks, \*subextensions  
> > \*added by those dependencies are not being replaced. The subextension  
> > addition logic happens in the container \*\_\_init\_\_\*  
> > \<[https://github.com/onefinestay/nameko/blob/master/nameko/containers.py#L160&gt;](https://github.com/onefinestay/nameko/blob/master/nameko/containers.py#L160&gt;),  
> > which happens before both \*worker\_factory\* and \*replace\_dependencies\* replace  
> > the \*RpcProxy\* dependencies with mocks. Therefore, \*RpcProxy\* dependencies  
> > add a QueueConsumer  
> > \<[https://github.com/onefinestay/nameko/blob/master/nameko/rpc.py#L40&gt](https://github.com/onefinestay/nameko/blob/master/nameko/rpc.py#L40&gt); dependencies  
> > to \*container.subextensions\* which triggers the RabbitMQ connection.
> > 
> > For now, I've been able to work around this by manually overriding the  
> > class dependencies before I use \*container\_factory, \*like this\*:\*
> > 
> > ExampleService.rpc\_dependency\_1 = Mock()  
> > ExampleService.rpc\_dependency\_2 = Mock()  
> > container = container\_factory(ExampleService, web\_config)  
> > container.start()
> > 
> > However, this seems suboptimal.
> > 
> > Has anyone dealt with this before?

---

<div class="post-metadata">

**Author:** ![Jesse\_Pollak](https://avatars.discourse-cdn.com/v4/letter/j/b5a626/32.png) [@Jesse\_Pollak](https://discourse.nameko.io/u/Jesse_Pollak)\
**Post date:** [August 10, 2016, 6:33pm UTC](https://discourse.nameko.io/t/running-tests-without-rabbitmq-when-using-worker-factory-and-container-factory/138/3 "2016-08-10T18:33:08Z")

</div>

Right now, we're primarily using container\_factory to test HTTP responses  
from HTTP services. In particular, we're creating a customer  
HTTPRequestHandler that processes exceptions and transforms them into  
standard responses and are testing that this works correctly. Is there a  
way to do HTTP-level testing with the worker\_factory?

Thanks David!  
Jesse

> **···**
>
> On Wednesday, August 10, 2016 at 1:21:13 AM UTC-7, David Szotten wrote:
> 
> > Hi
> > 
> > Could you give some more detail about why/when you are using the  
> > container\_factory over the worker\_factory? If you want everything mocked  
> > out it sounds like the worker\_factory (which won't need rabbitmq) is a  
> > better choice. The container factory is for when you want to simulate more  
> > realistic conditions, and so defaults to hosting real versions of everything
> > 
> > If you need only a few select dependencies, say a db connection, one  
> > option is to still use the worker\_factory, and manually inject the db, as  
> > in  
> > [https://github.com/onefinestay/nameko/blob/master/docs/examples/testing/alternative\_dependency\_unit\_test.py#L52](https://github.com/onefinestay/nameko/blob/master/docs/examples/testing/alternative_dependency_unit_test.py#L52)
> > 
> > Best,  
> > David
> > 
> > On Wednesday, 10 August 2016 08:21:48 UTC+1, je...@pollak.io wrote:
> > 
> > > Hi! We're writing tests for our services that test their high level  
> > > functionality using \*worker\_factory\* and \*container\_factory\*. When we  
> > > use \*worker\_factory\*, all the service's dependencies are replaced with  
> > > mock dependencies as specified in the docstring  
> > > \<[https://github.com/onefinestay/nameko/blob/master/nameko/testing/services.py#L141&gt;\](https://github.com/onefinestay/nameko/blob/master/nameko/testing/services.py#L141&gt;%5C).  
> > > When we use \*container\_factory\*, we are manually mocking all services  
> > > using the replace\_dependencies helper  
> > > \<[https://github.com/onefinestay/nameko/blob/master/nameko/testing/services.py#L229&gt;](https://github.com/onefinestay/nameko/blob/master/nameko/testing/services.py#L229&gt;),  
> > > which replaces the dependencies with mocks. Many of these are \*RpcProxy\* dependencies  
> > > that communicate with other services, but given the specified functionality  
> > > of \*worker\_factory\* and \*replace\_dependencies, \*we expect these  
> > > rpc-level dependencies to be mocked and for there to be no need for an  
> > > active RabbitMQ cluster.
> > > 
> > > \*Unfortunately, even though all of our dependencies are mocked, when we  
> > > try to run our tests, the tests error out on container.start() with a  
> > > failed connection to RabbitMQ.\*
> > > 
> > > Digging into the code a bit, I believe I've found the reason, but I  
> > > wanted to make sure I'm understanding.
> > > 
> > > I believe that while the \*dependencies\* are being replaced with mocks, \*subextensions  
> > > \*added by those dependencies are not being replaced. The subextension  
> > > addition logic happens in the container \*\_\_init\_\_\*  
> > > \<[https://github.com/onefinestay/nameko/blob/master/nameko/containers.py#L160&gt;](https://github.com/onefinestay/nameko/blob/master/nameko/containers.py#L160&gt;),  
> > > which happens before both \*worker\_factory\* and \*replace\_dependencies\* replace  
> > > the \*RpcProxy\* dependencies with mocks. Therefore, \*RpcProxy\* dependencies  
> > > add a QueueConsumer  
> > > \<[https://github.com/onefinestay/nameko/blob/master/nameko/rpc.py#L40&gt](https://github.com/onefinestay/nameko/blob/master/nameko/rpc.py#L40&gt); dependencies  
> > > to \*container.subextensions\* which triggers the RabbitMQ connection.
> > > 
> > > For now, I've been able to work around this by manually overriding the  
> > > class dependencies before I use \*container\_factory, \*like this\*:\*
> > > 
> > > ExampleService.rpc\_dependency\_1 = Mock()  
> > > ExampleService.rpc\_dependency\_2 = Mock()  
> > > container = container\_factory(ExampleService, web\_config)  
> > > container.start()
> > > 
> > > However, this seems suboptimal.
> > > 
> > > Has anyone dealt with this before?

---

<div class="post-metadata">

**Author:** ![David\_Szotten](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.nameko.io/david_szotten/32/6_2.png) [@David\_Szotten](https://discourse.nameko.io/u/David_Szotten)\
**Post date:** [August 10, 2016, 8:50pm UTC](https://discourse.nameko.io/t/running-tests-without-rabbitmq-when-using-worker-factory-and-container-factory/138/4 "2016-08-10T20:50:48Z")

</div>

Ah, interesting!

Some of the core nameko developers recently met up to discuss roadmaps and  
the eco system in general (notes will be posted to the mailing list once  
written up). One of the issues we discussed was scaling back the extent to  
which nameko assumes amqp. When nameko started out, there was no http  
extension, and amqp was so core to everything at onefinestay that we  
probably didn't consider the use case you brought up. we definitely need to  
though!

i need to have a think about the best way to handle this, but in the mean  
time, i threw together a workaround for  
you: [strip amqp dependencies from nameko class · GitHub](https://gist.github.com/davidszotten/d43e2d9225738a4efb7d25089196439b)

the core idea is a function `strip_rpc(service_class)` which dynamically  
creates a subclass of the service with all rpc entrypoints and proxy  
dependencies overridden by None.

the gist includes a test which uses the container\_factory to host a  
stripped version of the example service and successfully makes a http call  
without any rabbit broker required

best,  
David

> **···**
>
> On Wednesday, 10 August 2016 19:33:08 UTC+1, je...@pollak.io wrote:
> 
> > Right now, we're primarily using container\_factory to test HTTP responses  
> > from HTTP services. In particular, we're creating a customer  
> > HTTPRequestHandler that processes exceptions and transforms them into  
> > standard responses and are testing that this works correctly. Is there a  
> > way to do HTTP-level testing with the worker\_factory?
> > 
> > Thanks David!  
> > Jesse
> > 
> > On Wednesday, August 10, 2016 at 1:21:13 AM UTC-7, David Szotten wrote:
> > 
> > > Hi
> > > 
> > > Could you give some more detail about why/when you are using the  
> > > container\_factory over the worker\_factory? If you want everything mocked  
> > > out it sounds like the worker\_factory (which won't need rabbitmq) is a  
> > > better choice. The container factory is for when you want to simulate more  
> > > realistic conditions, and so defaults to hosting real versions of everything
> > > 
> > > If you need only a few select dependencies, say a db connection, one  
> > > option is to still use the worker\_factory, and manually inject the db, as  
> > > in  
> > > [https://github.com/onefinestay/nameko/blob/master/docs/examples/testing/alternative\_dependency\_unit\_test.py#L52](https://github.com/onefinestay/nameko/blob/master/docs/examples/testing/alternative_dependency_unit_test.py#L52)
> > > 
> > > Best,  
> > > David
> > > 
> > > On Wednesday, 10 August 2016 08:21:48 UTC+1, je...@pollak.io wrote:
> > > 
> > > > Hi! We're writing tests for our services that test their high level  
> > > > functionality using \*worker\_factory\* and \*container\_factory\*. When we  
> > > > use \*worker\_factory\*, all the service's dependencies are replaced with  
> > > > mock dependencies as specified in the docstring  
> > > > \<[https://github.com/onefinestay/nameko/blob/master/nameko/testing/services.py#L141&gt;\](https://github.com/onefinestay/nameko/blob/master/nameko/testing/services.py#L141&gt;%5C).  
> > > > When we use \*container\_factory\*, we are manually mocking all services  
> > > > using the replace\_dependencies helper  
> > > > \<[https://github.com/onefinestay/nameko/blob/master/nameko/testing/services.py#L229&gt;](https://github.com/onefinestay/nameko/blob/master/nameko/testing/services.py#L229&gt;),  
> > > > which replaces the dependencies with mocks. Many of these are \*RpcProxy\* dependencies  
> > > > that communicate with other services, but given the specified functionality  
> > > > of \*worker\_factory\* and \*replace\_dependencies, \*we expect these  
> > > > rpc-level dependencies to be mocked and for there to be no need for an  
> > > > active RabbitMQ cluster.
> > > > 
> > > > \*Unfortunately, even though all of our dependencies are mocked, when we  
> > > > try to run our tests, the tests error out on container.start() with a  
> > > > failed connection to RabbitMQ.\*
> > > > 
> > > > Digging into the code a bit, I believe I've found the reason, but I  
> > > > wanted to make sure I'm understanding.
> > > > 
> > > > I believe that while the \*dependencies\* are being replaced with mocks, \*subextensions  
> > > > \*added by those dependencies are not being replaced. The subextension  
> > > > addition logic happens in the container \*\_\_init\_\_\*  
> > > > \<[https://github.com/onefinestay/nameko/blob/master/nameko/containers.py#L160&gt;](https://github.com/onefinestay/nameko/blob/master/nameko/containers.py#L160&gt;),  
> > > > which happens before both \*worker\_factory\* and \*replace\_dependencies\* replace  
> > > > the \*RpcProxy\* dependencies with mocks. Therefore, \*RpcProxy\* dependencies  
> > > > add a QueueConsumer  
> > > > \<[https://github.com/onefinestay/nameko/blob/master/nameko/rpc.py#L40&gt](https://github.com/onefinestay/nameko/blob/master/nameko/rpc.py#L40&gt); dependencies  
> > > > to \*container.subextensions\* which triggers the RabbitMQ connection.
> > > > 
> > > > For now, I've been able to work around this by manually overriding the  
> > > > class dependencies before I use \*container\_factory, \*like this\*:\*
> > > > 
> > > > ExampleService.rpc\_dependency\_1 = Mock()  
> > > > ExampleService.rpc\_dependency\_2 = Mock()  
> > > > container = container\_factory(ExampleService, web\_config)  
> > > > container.start()
> > > > 
> > > > However, this seems suboptimal.
> > > > 
> > > > Has anyone dealt with this before?

---

<div class="post-metadata">

**Author:** ![Jesse\_Pollak](https://avatars.discourse-cdn.com/v4/letter/j/b5a626/32.png) [@Jesse\_Pollak](https://discourse.nameko.io/u/Jesse_Pollak)\
**Post date:** [August 11, 2016, 12:13am UTC](https://discourse.nameko.io/t/running-tests-without-rabbitmq-when-using-worker-factory-and-container-factory/138/5 "2016-08-11T00:13:56Z")

</div>

That makes total sense - and thanks for the gist! That's a much more  
sophisticated way of doing what I was previously doing manually 🙂

Just a note: we've started using nameko pretty heavily at Clef  
\<[http://getclef.com](http://getclef.com)\> and are really enjoying it (Mark, Grace and I -- all  
recent mailers -- all work there)! Thanks for all your hard work so far and  
I hope we're able to contribute back going forward.

Jesse

> **···**
>
> On Wednesday, August 10, 2016 at 1:50:48 PM UTC-7, David Szotten wrote:
> 
> > Ah, interesting!
> > 
> > Some of the core nameko developers recently met up to discuss roadmaps and  
> > the eco system in general (notes will be posted to the mailing list once  
> > written up). One of the issues we discussed was scaling back the extent to  
> > which nameko assumes amqp. When nameko started out, there was no http  
> > extension, and amqp was so core to everything at onefinestay that we  
> > probably didn't consider the use case you brought up. we definitely need to  
> > though!
> > 
> > i need to have a think about the best way to handle this, but in the mean  
> > time, i threw together a workaround for you:  
> > [strip amqp dependencies from nameko class · GitHub](https://gist.github.com/davidszotten/d43e2d9225738a4efb7d25089196439b)
> > 
> > the core idea is a function `strip_rpc(service_class)` which dynamically  
> > creates a subclass of the service with all rpc entrypoints and proxy  
> > dependencies overridden by None.
> > 
> > the gist includes a test which uses the container\_factory to host a  
> > stripped version of the example service and successfully makes a http call  
> > without any rabbit broker required
> > 
> > best,  
> > David
> > 
> > On Wednesday, 10 August 2016 19:33:08 UTC+1, je...@pollak.io wrote:
> > 
> > > Right now, we're primarily using container\_factory to test HTTP responses  
> > > from HTTP services. In particular, we're creating a customer  
> > > HTTPRequestHandler that processes exceptions and transforms them into  
> > > standard responses and are testing that this works correctly. Is there a  
> > > way to do HTTP-level testing with the worker\_factory?
> > > 
> > > Thanks David!  
> > > Jesse
> > > 
> > > On Wednesday, August 10, 2016 at 1:21:13 AM UTC-7, David Szotten wrote:
> > > 
> > > > Hi
> > > > 
> > > > Could you give some more detail about why/when you are using the  
> > > > container\_factory over the worker\_factory? If you want everything mocked  
> > > > out it sounds like the worker\_factory (which won't need rabbitmq) is a  
> > > > better choice. The container factory is for when you want to simulate more  
> > > > realistic conditions, and so defaults to hosting real versions of everything
> > > > 
> > > > If you need only a few select dependencies, say a db connection, one  
> > > > option is to still use the worker\_factory, and manually inject the db, as  
> > > > in  
> > > > [https://github.com/onefinestay/nameko/blob/master/docs/examples/testing/alternative\_dependency\_unit\_test.py#L52](https://github.com/onefinestay/nameko/blob/master/docs/examples/testing/alternative_dependency_unit_test.py#L52)
> > > > 
> > > > Best,  
> > > > David
> > > > 
> > > > On Wednesday, 10 August 2016 08:21:48 UTC+1, je...@pollak.io wrote:
> > > > 
> > > > > Hi! We're writing tests for our services that test their high level  
> > > > > functionality using \*worker\_factory\* and \*container\_factory\*. When we  
> > > > > use \*worker\_factory\*, all the service's dependencies are replaced with  
> > > > > mock dependencies as specified in the docstring  
> > > > > \<[https://github.com/onefinestay/nameko/blob/master/nameko/testing/services.py#L141&gt;\](https://github.com/onefinestay/nameko/blob/master/nameko/testing/services.py#L141&gt;%5C).  
> > > > > When we use \*container\_factory\*, we are manually mocking all services  
> > > > > using the replace\_dependencies helper  
> > > > > \<[https://github.com/onefinestay/nameko/blob/master/nameko/testing/services.py#L229&gt;](https://github.com/onefinestay/nameko/blob/master/nameko/testing/services.py#L229&gt;),  
> > > > > which replaces the dependencies with mocks. Many of these are  
> > > > > \*RpcProxy\* dependencies that communicate with other services, but  
> > > > > given the specified functionality of \*worker\_factory\* and \*replace\_dependencies,  
> > > > > \*we expect these rpc-level dependencies to be mocked and for there to  
> > > > > be no need for an active RabbitMQ cluster.
> > > > > 
> > > > > \*Unfortunately, even though all of our dependencies are mocked, when we  
> > > > > try to run our tests, the tests error out on container.start() with a  
> > > > > failed connection to RabbitMQ.\*
> > > > > 
> > > > > Digging into the code a bit, I believe I've found the reason, but I  
> > > > > wanted to make sure I'm understanding.
> > > > > 
> > > > > I believe that while the \*dependencies\* are being replaced with mocks, \*subextensions  
> > > > > \*added by those dependencies are not being replaced. The subextension  
> > > > > addition logic happens in the container \*\_\_init\_\_\*  
> > > > > \<[https://github.com/onefinestay/nameko/blob/master/nameko/containers.py#L160&gt;](https://github.com/onefinestay/nameko/blob/master/nameko/containers.py#L160&gt;),  
> > > > > which happens before both \*worker\_factory\* and \*replace\_dependencies\* replace  
> > > > > the \*RpcProxy\* dependencies with mocks. Therefore, \*RpcProxy\* dependencies  
> > > > > add a QueueConsumer  
> > > > > \<[https://github.com/onefinestay/nameko/blob/master/nameko/rpc.py#L40&gt](https://github.com/onefinestay/nameko/blob/master/nameko/rpc.py#L40&gt); dependencies  
> > > > > to \*container.subextensions\* which triggers the RabbitMQ connection.
> > > > > 
> > > > > For now, I've been able to work around this by manually overriding the  
> > > > > class dependencies before I use \*container\_factory, \*like this\*:\*
> > > > > 
> > > > > ExampleService.rpc\_dependency\_1 = Mock()  
> > > > > ExampleService.rpc\_dependency\_2 = Mock()  
> > > > > container = container\_factory(ExampleService, web\_config)  
> > > > > container.start()
> > > > > 
> > > > > However, this seems suboptimal.
> > > > > 
> > > > > Has anyone dealt with this before?

---

<div class="post-metadata">

**Author:** ![David\_Szotten](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.nameko.io/david_szotten/32/6_2.png) [@David\_Szotten](https://discourse.nameko.io/u/David_Szotten)\
**Post date:** [August 11, 2016, 6:38am UTC](https://discourse.nameko.io/t/running-tests-without-rabbitmq-when-using-worker-factory-and-container-factory/138/6 "2016-08-11T06:38:32Z")

</div>

glad to help

the usual caveats about using undocumented internals apply. hopefully we'll  
find a way to better support this use-case in the future

d

> **···**
>
> On Thursday, 11 August 2016 01:13:56 UTC+1, je...@pollak.io wrote:
> 
> > That makes total sense - and thanks for the gist! That's a much more  
> > sophisticated way of doing what I was previously doing manually 🙂
> > 
> > Just a note: we've started using nameko pretty heavily at Clef  
> > \<[http://getclef.com](http://getclef.com)\> and are really enjoying it (Mark, Grace and I -- all  
> > recent mailers -- all work there)! Thanks for all your hard work so far and  
> > I hope we're able to contribute back going forward.
> > 
> > Jesse
> > 
> > On Wednesday, August 10, 2016 at 1:50:48 PM UTC-7, David Szotten wrote:
> > 
> > > Ah, interesting!
> > > 
> > > Some of the core nameko developers recently met up to discuss roadmaps  
> > > and the eco system in general (notes will be posted to the mailing list  
> > > once written up). One of the issues we discussed was scaling back the  
> > > extent to which nameko assumes amqp. When nameko started out, there was no  
> > > http extension, and amqp was so core to everything at onefinestay that we  
> > > probably didn't consider the use case you brought up. we definitely need to  
> > > though!
> > > 
> > > i need to have a think about the best way to handle this, but in the mean  
> > > time, i threw together a workaround for you:  
> > > [strip amqp dependencies from nameko class · GitHub](https://gist.github.com/davidszotten/d43e2d9225738a4efb7d25089196439b)
> > > 
> > > the core idea is a function `strip_rpc(service_class)` which dynamically  
> > > creates a subclass of the service with all rpc entrypoints and proxy  
> > > dependencies overridden by None.
> > > 
> > > the gist includes a test which uses the container\_factory to host a  
> > > stripped version of the example service and successfully makes a http call  
> > > without any rabbit broker required
> > > 
> > > best,  
> > > David
> > > 
> > > On Wednesday, 10 August 2016 19:33:08 UTC+1, je...@pollak.io wrote:
> > > 
> > > > Right now, we're primarily using container\_factory to test HTTP  
> > > > responses from HTTP services. In particular, we're creating a customer  
> > > > HTTPRequestHandler that processes exceptions and transforms them into  
> > > > standard responses and are testing that this works correctly. Is there a  
> > > > way to do HTTP-level testing with the worker\_factory?
> > > > 
> > > > Thanks David!  
> > > > Jesse
> > > > 
> > > > On Wednesday, August 10, 2016 at 1:21:13 AM UTC-7, David Szotten wrote:
> > > > 
> > > > > Hi
> > > > > 
> > > > > Could you give some more detail about why/when you are using the  
> > > > > container\_factory over the worker\_factory? If you want everything mocked  
> > > > > out it sounds like the worker\_factory (which won't need rabbitmq) is a  
> > > > > better choice. The container factory is for when you want to simulate more  
> > > > > realistic conditions, and so defaults to hosting real versions of everything
> > > > > 
> > > > > If you need only a few select dependencies, say a db connection, one  
> > > > > option is to still use the worker\_factory, and manually inject the db, as  
> > > > > in  
> > > > > [https://github.com/onefinestay/nameko/blob/master/docs/examples/testing/alternative\_dependency\_unit\_test.py#L52](https://github.com/onefinestay/nameko/blob/master/docs/examples/testing/alternative_dependency_unit_test.py#L52)
> > > > > 
> > > > > Best,  
> > > > > David
> > > > > 
> > > > > On Wednesday, 10 August 2016 08:21:48 UTC+1, je...@pollak.io wrote:
> > > > > 
> > > > > > Hi! We're writing tests for our services that test their high level  
> > > > > > functionality using \*worker\_factory\* and \*container\_factory\*. When we  
> > > > > > use \*worker\_factory\*, all the service's dependencies are replaced  
> > > > > > with mock dependencies as specified in the docstring  
> > > > > > \<[https://github.com/onefinestay/nameko/blob/master/nameko/testing/services.py#L141&gt;\](https://github.com/onefinestay/nameko/blob/master/nameko/testing/services.py#L141&gt;%5C).  
> > > > > > When we use \*container\_factory\*, we are manually mocking all services  
> > > > > > using the replace\_dependencies helper  
> > > > > > \<[https://github.com/onefinestay/nameko/blob/master/nameko/testing/services.py#L229&gt;](https://github.com/onefinestay/nameko/blob/master/nameko/testing/services.py#L229&gt;),  
> > > > > > which replaces the dependencies with mocks. Many of these are  
> > > > > > \*RpcProxy\* dependencies that communicate with other services, but  
> > > > > > given the specified functionality of \*worker\_factory\* and \*replace\_dependencies,  
> > > > > > \*we expect these rpc-level dependencies to be mocked and for there to  
> > > > > > be no need for an active RabbitMQ cluster.
> > > > > > 
> > > > > > \*Unfortunately, even though all of our dependencies are mocked, when  
> > > > > > we try to run our tests, the tests error out on container.start() with a  
> > > > > > failed connection to RabbitMQ.\*
> > > > > > 
> > > > > > Digging into the code a bit, I believe I've found the reason, but I  
> > > > > > wanted to make sure I'm understanding.
> > > > > > 
> > > > > > I believe that while the \*dependencies\* are being replaced with  
> > > > > > mocks, \*subextensions \*added by those dependencies are not being  
> > > > > > replaced. The subextension addition logic happens in the container  
> > > > > > \*\_\_init\_\_\*  
> > > > > > \<[https://github.com/onefinestay/nameko/blob/master/nameko/containers.py#L160&gt;](https://github.com/onefinestay/nameko/blob/master/nameko/containers.py#L160&gt;),  
> > > > > > which happens before both \*worker\_factory\* and \*replace\_dependencies\* replace  
> > > > > > the \*RpcProxy\* dependencies with mocks. Therefore, \*RpcProxy\* dependencies  
> > > > > > add a QueueConsumer  
> > > > > > \<[https://github.com/onefinestay/nameko/blob/master/nameko/rpc.py#L40&gt](https://github.com/onefinestay/nameko/blob/master/nameko/rpc.py#L40&gt); dependencies  
> > > > > > to \*container.subextensions\* which triggers the RabbitMQ connection.
> > > > > > 
> > > > > > For now, I've been able to work around this by manually overriding the  
> > > > > > class dependencies before I use \*container\_factory, \*like this\*:\*
> > > > > > 
> > > > > > ExampleService.rpc\_dependency\_1 = Mock()  
> > > > > > ExampleService.rpc\_dependency\_2 = Mock()  
> > > > > > container = container\_factory(ExampleService, web\_config)  
> > > > > > container.start()
> > > > > > 
> > > > > > However, this seems suboptimal.
> > > > > > 
> > > > > > Has anyone dealt with this before?

---

<div class="post-metadata">

**Author:** ![mattbennett](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.nameko.io/mattbennett/32/13_2.png) [@mattbennett](https://discourse.nameko.io/u/mattbennett)\
**Post date:** [August 11, 2016, 11:08am UTC](https://discourse.nameko.io/t/running-tests-without-rabbitmq-when-using-worker-factory-and-container-factory/138/7 "2016-08-11T11:08:16Z")

</div>

Hi Jesse,

Your diagnosis is completely correct. `replace_dependencies` mocks out the  
dependency providers only, and isn't clever enough to know about any  
subextensions that they may have created.

This is probably an oversight in the way sub-extensions are declared. The  
container discovers them by inspecting the "top-level" extensions, but then  
treats them all the same. The knowledge of who declared them is lost. I  
would rather `start()` and `stop()` only the top-level extensions and make  
them responsible for the sub-extensions they declare; then  
`replace_dependencies` could work in the way you expected. Unfortunately  
this change isn't compatible with current implementation of the  
QueueConsumer. The good news is that we also discussed refactoring this at  
the recent in-person meetup. As promised I will write up these notes soon.

It's great to hear about nameko being used at Clef! Thanks for sharing and  
I'm glad you're enjoying it 🙂

Matt.

> **···**
>
> On Thursday, August 11, 2016 at 2:38:32 PM UTC+8, David Szotten wrote:
> 
> > glad to help
> > 
> > the usual caveats about using undocumented internals apply. hopefully  
> > we'll find a way to better support this use-case in the future
> > 
> > d
> > 
> > On Thursday, 11 August 2016 01:13:56 UTC+1, je...@pollak.io wrote:
> > 
> > > That makes total sense - and thanks for the gist! That's a much more  
> > > sophisticated way of doing what I was previously doing manually 🙂
> > > 
> > > Just a note: we've started using nameko pretty heavily at Clef  
> > > \<[http://getclef.com](http://getclef.com)\> and are really enjoying it (Mark, Grace and I --  
> > > all recent mailers -- all work there)! Thanks for all your hard work so far  
> > > and I hope we're able to contribute back going forward.
> > > 
> > > Jesse
> > > 
> > > On Wednesday, August 10, 2016 at 1:50:48 PM UTC-7, David Szotten wrote:
> > > 
> > > > Ah, interesting!
> > > > 
> > > > Some of the core nameko developers recently met up to discuss roadmaps  
> > > > and the eco system in general (notes will be posted to the mailing list  
> > > > once written up). One of the issues we discussed was scaling back the  
> > > > extent to which nameko assumes amqp. When nameko started out, there was no  
> > > > http extension, and amqp was so core to everything at onefinestay that we  
> > > > probably didn't consider the use case you brought up. we definitely need to  
> > > > though!
> > > > 
> > > > i need to have a think about the best way to handle this, but in the  
> > > > mean time, i threw together a workaround for you:  
> > > > [strip amqp dependencies from nameko class · GitHub](https://gist.github.com/davidszotten/d43e2d9225738a4efb7d25089196439b)
> > > > 
> > > > the core idea is a function `strip_rpc(service_class)` which dynamically  
> > > > creates a subclass of the service with all rpc entrypoints and proxy  
> > > > dependencies overridden by None.
> > > > 
> > > > the gist includes a test which uses the container\_factory to host a  
> > > > stripped version of the example service and successfully makes a http call  
> > > > without any rabbit broker required
> > > > 
> > > > best,  
> > > > David
> > > > 
> > > > On Wednesday, 10 August 2016 19:33:08 UTC+1, je...@pollak.io wrote:
> > > > 
> > > > > Right now, we're primarily using container\_factory to test HTTP  
> > > > > responses from HTTP services. In particular, we're creating a customer  
> > > > > HTTPRequestHandler that processes exceptions and transforms them into  
> > > > > standard responses and are testing that this works correctly. Is there a  
> > > > > way to do HTTP-level testing with the worker\_factory?
> > > > > 
> > > > > Thanks David!  
> > > > > Jesse
> > > > > 
> > > > > On Wednesday, August 10, 2016 at 1:21:13 AM UTC-7, David Szotten wrote:
> > > > > 
> > > > > > Hi
> > > > > > 
> > > > > > Could you give some more detail about why/when you are using the  
> > > > > > container\_factory over the worker\_factory? If you want everything mocked  
> > > > > > out it sounds like the worker\_factory (which won't need rabbitmq) is a  
> > > > > > better choice. The container factory is for when you want to simulate more  
> > > > > > realistic conditions, and so defaults to hosting real versions of everything
> > > > > > 
> > > > > > If you need only a few select dependencies, say a db connection, one  
> > > > > > option is to still use the worker\_factory, and manually inject the db, as  
> > > > > > in  
> > > > > > [https://github.com/onefinestay/nameko/blob/master/docs/examples/testing/alternative\_dependency\_unit\_test.py#L52](https://github.com/onefinestay/nameko/blob/master/docs/examples/testing/alternative_dependency_unit_test.py#L52)
> > > > > > 
> > > > > > Best,  
> > > > > > David
> > > > > > 
> > > > > > On Wednesday, 10 August 2016 08:21:48 UTC+1, je...@pollak.io wrote:
> > > > > > 
> > > > > > > Hi! We're writing tests for our services that test their high level  
> > > > > > > functionality using \*worker\_factory\* and \*container\_factory\*. When  
> > > > > > > we use \*worker\_factory\*, all the service's dependencies are replaced  
> > > > > > > with mock dependencies as specified in the docstring  
> > > > > > > \<[https://github.com/onefinestay/nameko/blob/master/nameko/testing/services.py#L141&gt;\](https://github.com/onefinestay/nameko/blob/master/nameko/testing/services.py#L141&gt;%5C).  
> > > > > > > When we use \*container\_factory\*, we are manually mocking all  
> > > > > > > services using the replace\_dependencies helper  
> > > > > > > \<[https://github.com/onefinestay/nameko/blob/master/nameko/testing/services.py#L229&gt;](https://github.com/onefinestay/nameko/blob/master/nameko/testing/services.py#L229&gt;),  
> > > > > > > which replaces the dependencies with mocks. Many of these are  
> > > > > > > \*RpcProxy\* dependencies that communicate with other services, but  
> > > > > > > given the specified functionality of \*worker\_factory\* and \*replace\_dependencies,  
> > > > > > > \*we expect these rpc-level dependencies to be mocked and for there  
> > > > > > > to be no need for an active RabbitMQ cluster.
> > > > > > > 
> > > > > > > \*Unfortunately, even though all of our dependencies are mocked, when  
> > > > > > > we try to run our tests, the tests error out on container.start() with a  
> > > > > > > failed connection to RabbitMQ.\*
> > > > > > > 
> > > > > > > Digging into the code a bit, I believe I've found the reason, but I  
> > > > > > > wanted to make sure I'm understanding.
> > > > > > > 
> > > > > > > I believe that while the \*dependencies\* are being replaced with  
> > > > > > > mocks, \*subextensions \*added by those dependencies are not being  
> > > > > > > replaced. The subextension addition logic happens in the container  
> > > > > > > \*\_\_init\_\_\*  
> > > > > > > \<[https://github.com/onefinestay/nameko/blob/master/nameko/containers.py#L160&gt;](https://github.com/onefinestay/nameko/blob/master/nameko/containers.py#L160&gt;),  
> > > > > > > which happens before both \*worker\_factory\* and \*replace\_dependencies\* replace  
> > > > > > > the \*RpcProxy\* dependencies with mocks. Therefore, \*RpcProxy\* dependencies  
> > > > > > > add a QueueConsumer  
> > > > > > > \<[https://github.com/onefinestay/nameko/blob/master/nameko/rpc.py#L40&gt](https://github.com/onefinestay/nameko/blob/master/nameko/rpc.py#L40&gt); dependencies  
> > > > > > > to \*container.subextensions\* which triggers the RabbitMQ connection.
> > > > > > > 
> > > > > > > For now, I've been able to work around this by manually overriding  
> > > > > > > the class dependencies before I use \*container\_factory, \*like this\*:\*
> > > > > > > 
> > > > > > > ExampleService.rpc\_dependency\_1 = Mock()  
> > > > > > > ExampleService.rpc\_dependency\_2 = Mock()  
> > > > > > > container = container\_factory(ExampleService, web\_config)  
> > > > > > > container.start()
> > > > > > > 
> > > > > > > However, this seems suboptimal.
> > > > > > > 
> > > > > > > Has anyone dealt with this before?