[openstack-community] [Openstack] [NOVA] nova image-list error class 'glanceclient.exc.CommunicationError'> (HTTP 500)
Rajiv Duggal
rduggal at dexsystems.com
Mon Jan 11 22:38:33 UTC 2016
Hi,
I am a newbie to the this list. After a lot of attempts to get this working wanted to try out the mailing list for ideas. Please review this email in detail before suggesting to check password or other basic stuff.
My conlusion this far is that my environment on liberty is running glance v2. Nova is making calls to v2 of nova api at port 8794. This returns a error for image-list. Not sure if it is calling v1 of glance or not.
Glance v1 api is erroring out.
Looking for some pointers to below.
How do I add more debug to ensure nova is using glance api v2. I have seen some articles that state nova uses v1 api of glance, but not sure if that applies to liberty release.
If yes then how do I get v1 glance api working on the box.
See below for all the debug and issue details. Any help or response is greatly appreciated.
nova image-list returns http 500 'glanceclient.exc.CommunicationError'. Glance image-list works. Openstack image list works. Nova service-list works. cinder list works.
Glance v2 api works and is able to create images. Cinder is able to create volumes from the glance images.
All configuration files are enabled to debug and verbose . Nothing showing in the cinder or glance debug logs that would provide more details.
Glance is using version 2. IF I try version 1 api glance will error out. Does glance api v1 works with liberty. How do I enable this. My storage backend is ceph.
RBD is able to communicate to all volumes and all user ids are set correctly and have rrx for stores for glance, cinder and nova. Keyrings are present.
Not seeing any errors with keystone .
Controller box has haproxy,mariadb, glance and cinder. nova and neutron are installed on a separate box. Passwords in the mysql database for glance,cinder,nova match what is in the config file.
Since I am using haproxy for HA the end point ports are non default. Services themselves are running on the default ports on each box.
For debug isolation purposes only using 1 controller (glance,cinder,keystone, haproxy) and 1 compute (nova and neutron)
# openstack image list
+--------------------------------------+--------+
| ID | Name |
+--------------------------------------+--------+
| f4040cc9-a2fa-45d5-9da3-52ffdde998a8 | cirros |
| de815da9-135a-4f8d-99ff-9e954672d775 | cirros |
# glance image-list
+--------------------------------------+--------+
| ID | Name |
+--------------------------------------+--------+
| de815da9-135a-4f8d-99ff-9e954672d775 | cirros |
| f4040cc9-a2fa-45d5-9da3-52ffdde998a8 | cirros |
+--------------------------------------+--------+
# nova service-list
+----+------------------+----------+----------+---------+-------+----------------------------+-----------------+
| Id | Binary | Host | Zone | Status | State | Updated_at | Disabled Reason |
+----+------------------+----------+----------+---------+-------+----------------------------+-----------------+
| 3 | nova-cert | ostackn1 | internal | enabled | up | 2016-01-11T19:29:52.000000 | - |
| 5 | nova-consoleauth | ostackn1 | internal | enabled | up | 2016-01-11T19:29:56.000000 | - |
| 7 | nova-conductor | ostackn1 | internal | enabled | up | 2016-01-11T19:29:49.000000 | - |
| 27 | nova-scheduler | ostackn1 | internal | enabled | up | 2016-01-11T19:29:51.000000 | - |
| 29 | nova-compute | ostackn1 | nova | enabled | up | 2016-01-11T19:29:54.000000 | - |
| 31 | nova-network | ostackn1 | internal | enabled | up | 2016-01-11T19:29:49.000000 | - |
+----+------------------+----------+----------+---------+-------+----------------------------+-----------------+
root at ostackn1:~# nova --debug image-list
DEBUG (session:198) REQ: curl -g -i -X GET http://10.1.4.54:35358/v2.0 -H "Accept: application/json" -H "User-Agent: python-keystoneclient"
INFO (connectionpool:205) Starting new HTTP connection (1): 10.1.4.54
DEBUG (connectionpool:385) "GET /v2.0 HTTP/1.1" 200 336
DEBUG (session:215) RESP: [200] Content-Length: 336 Vary: X-Auth-Token X-Distribution: Ubuntu Date: Mon, 11 Jan 2016 19:26:51 GMT Content-Type: application/json X-Openstack-Request-Id: req-e5ff4279-4632-4ae2-9f4f-af6cfdcb6727
RESP BODY: {"version": {"status": "stable", "updated": "2014-04-17T00:00:00Z", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v2.0+json"}], "id": "v2.0", "links": [{"href": "http://10.1.4.54:35358/v2.0/", "rel": "self"}, {"href": "http://docs.openstack.org/", "type": "text/html", "rel": "describedby"}]}}
DEBUG (v2:86) Making authentication request to http://10.1.4.54:35358/v2.0/tokens
DEBUG (connectionpool:385) "POST /v2.0/tokens HTTP/1.1" 200 3066
DEBUG (session:198) REQ: curl -g -i -X GET http://10.1.4.54:8794/v2/ -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}b62d2f18c957d5cebd47eb593f4a55f8ccec82a8"
INFO (connectionpool:205) Starting new HTTP connection (1): 10.1.4.54
DEBUG (connectionpool:385) "GET /v2/ HTTP/1.1" 200 374
DEBUG (session:215) RESP: [200] Date: Mon, 11 Jan 2016 19:26:53 GMT Content-Length: 374 Content-Type: application/json X-Compute-Request-Id: req-0a444962-ebec-4994-bd82-d00bacbfca60
RESP BODY: {"version": {"status": "SUPPORTED", "updated": "2011-01-21T11:33:21Z", "links": [{"href": "http://10.1.4.54:8794/v2/", "rel": "self"}, {"href": "http://docs.openstack.org/", "type": "text/html", "rel": "describedby"}], "min_version": "", "version": "", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.compute+json;version=2"}], "id": "v2.0"}}
DEBUG (session:198) REQ: curl -g -i -X GET http://10.1.4.54:8794/v2/ab601ee07c3a4ed293dc04f5fa1d90a1/images/detail -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}b62d2f18c957d5cebd47eb593f4a55f8ccec82a8"
DEBUG (connectionpool:385) "GET /v2/ab601ee07c3a4ed293dc04f5fa1d90a1/images/detail HTTP/1.1" 500 209
DEBUG (session:215) RESP: [500] Date: Mon, 11 Jan 2016 19:26:53 GMT Content-Length: 209 Content-Type: application/json; charset=UTF-8 X-Compute-Request-Id: req-ffab41dd-fb6b-40e0-b739-5705e7bd3b84
RESP BODY: {"computeFault": {"message": "Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.\n<class 'glanceclient.exc.CommunicationError'>", "code": 500}}
DEBUG (shell:905) Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'glanceclient.exc.CommunicationError'> (HTTP 500) (Request-ID: req-ffab41dd-fb6b-40e0-b739-5705e7bd3b84)
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/novaclient/shell.py", line 903, in main
OpenStackComputeShell().main(argv)
File "/usr/lib/python2.7/dist-packages/novaclient/shell.py", line 830, in main
args.func(self.cs, args)
File "/usr/lib/python2.7/dist-packages/novaclient/v2/shell.py", line 1115, in do_image_list
image_list = cs.images.list(limit=limit)
File "/usr/lib/python2.7/dist-packages/novaclient/v2/images.py", line 74, in list
return self._list('/images%s%s' % (detail, query), 'images')
File "/usr/lib/python2.7/dist-packages/novaclient/base.py", line 72, in _list
_resp, body = self.api.client.get(url)
File "/usr/lib/python2.7/dist-packages/keystoneclient/adapter.py", line 170, in get
return self.request(url, 'GET', **kwargs)
File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 93, in request
raise exceptions.from_response(resp, body, url, method)
ClientException: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'glanceclient.exc.CommunicationError'> (HTTP 500) (Request-ID: req-ffab41dd-fb6b-40e0-b739-5705e7bd3b84)
ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'glanceclient.exc.CommunicationError'> (HTTP 500) (Request-ID: req-ffab41dd-fb6b-40e0-b739-5705e7bd3b84)
root at ostackc1:~# glance --os-image-api-version 2 image-list
+--------------------------------------+--------+
| ID | Name |
+--------------------------------------+--------+
| de815da9-135a-4f8d-99ff-9e954672d775 | cirros |
| f4040cc9-a2fa-45d5-9da3-52ffdde998a8 | cirros |
+--------------------------------------+--------+
root at ostackc1:~# glance --debug --os-image-api-version 1 image-list
curl -g -i -X GET -H 'Accept-Encoding: gzip, deflate' -H 'Accept: */*' -H 'User-Agent: python-glanceclient' -H 'Connection: keep-alive' -H 'X-Auth-Token: {SHA1}318d44da7749a1b808ca32024fedc494a0f76133' -H 'Content-Type: application/octet-stream' http://10.1.4.54:9293/v1/images/detail?sort_key=name&sort_dir=asc&limit=20
Request returned failure status 500.
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/glanceclient/shell.py", line 700, in main
args.func(client, args)
File "/usr/lib/python2.7/dist-packages/glanceclient/v1/shell.py", line 123, in do_image_list
utils.print_list(images, columns)
File "/usr/lib/python2.7/dist-packages/glanceclient/common/utils.py", line 183, in print_list
for o in objs:
File "/usr/lib/python2.7/dist-packages/glanceclient/v1/images.py", line 254, in list
for image in paginate(params, return_request_id):
File "/usr/lib/python2.7/dist-packages/glanceclient/v1/images.py", line 238, in paginate
images, resp = self._list(url, "images")
File "/usr/lib/python2.7/dist-packages/glanceclient/v1/images.py", line 63, in _list
resp, body = self.client.get(url)
File "/usr/lib/python2.7/dist-packages/glanceclient/common/http.py", line 280, in get
return self._request('GET', url, **kwargs)
File "/usr/lib/python2.7/dist-packages/glanceclient/common/http.py", line 272, in _request
resp, body_iter = self._handle_response(resp)
File "/usr/lib/python2.7/dist-packages/glanceclient/common/http.py", line 93, in _handle_response
raise exc.from_response(resp, resp.content)
HTTPInternalServerError: 500 Internal Server Error: The server has either erred or is incapable of performing the requested operation. (HTTP 500)
500 Internal Server Error: The server has either erred or is incapable of performing the requested operation. (HTTP 500)
keystone endpoint-list
+----------------------------------+-----------+---------------------------------------------+---------------------------------------------+----------------------------------------+----------------------------------+
| id | region | publicurl | internalurl | adminurl | service_id |
+----------------------------------+-----------+---------------------------------------------+---------------------------------------------+----------------------------------------+----------------------------------+
| 045de6640536445da86969241286fb01 | regionOne | http://10.1.4.54:8081/v1/AUTH_%(tenant_id)s<http://10.1.4.54:8081/v1/AUTH_%25(tenant_id)s> | http://10.1.4.54:8081/v1/AUTH_%(tenant_id)s<http://10.1.4.54:8081/v1/AUTH_%25(tenant_id)s> | http://10.1.4.54:8081 | f3f2704480ea4b02bff61741e3ba734c |
| 0c399f7fd9f9456ea158679ac2d9e800 | regionOne | http://10.1.4.54:8794/v2/%(tenant_id)s<http://10.1.4.54:8794/v2/%25(tenant_id)s> | http://10.1.4.54:8794/v2/%(tenant_id)s<http://10.1.4.54:8794/v2/%25(tenant_id)s> | http://10.1.4.54:8794/v2/%(tenant_id)s<http://10.1.4.54:8794/v2/%25(tenant_id)s> | 6365cc8ba6c24c5fa624fc587e9d6614 |
| 1a8e073f90bf4450b0bb5fbc6d356b4a | regionOne | http://10.1.4.54:8777/v1/%(tenant_id)s<http://10.1.4.54:8777/v1/%25(tenant_id)s> | http://10.1.4.54:8777/v1/%(tenant_id)s<http://10.1.4.54:8777/v1/%25(tenant_id)s> | http://10.1.4.54:8777/v1/%(tenant_id)s<http://10.1.4.54:8777/v1/%25(tenant_id)s> | 8ff95f752a104f0f8f8e65ab02a550a2 |
| 33f8ec041a2c444d975317a9c74f5b7e | regionOne | http://10.1.4.54:9293 | http://10.1.4.54:9293 | http://10.1.4.54:9293 | 622081c852b4498cb71b6b94651f8577 |
| e6887352979c41b7aad8a0a77d67d80b | regionOne | http://10.1.4.54:5001/v2.0 | http://10.1.4.54:5001/v2.0 | http://10.1.4.54:35358/v2.0 | 2aeac7db1c154706ab2560d978073232 |
| ec164aed45c04499900098fa21ce9aac | regionOne | http://10.1.4.54:8777/v2/%(tenant_id)s<http://10.1.4.54:8777/v2/%25(tenant_id)s> | http://10.1.4.54:8777/v2/%(tenant_id)s<http://10.1.4.54:8777/v2/%25(tenant_id)s> | http://10.1.4.54:8777/v2/%(tenant_id)s<http://10.1.4.54:8777/v2/%25(tenant_id)s> | d857ec11a2414b95a5dc5b7f10cb7300 |
| f34cea0c5db04f17bde7b37e48d7372f | regionOne | http://10.1.4.54:9697 | http://10.1.4.54:9697 | http://10.1.4.54:9697 | eee2db07934f4353b13211375a5df6a5 |
| ff54d6bd114f4e58bb1f26cf63f1d8af | regionOne | http://10.1.4.54:8793/services/Cloud | http://10.1.4.54:8793/services/Cloud | http://10.1.4.54:8793/services/Cloud | 2ad2865f6ef747a39112b2b58be13a1f |
+----------------------------------+-----------+---------------------------------------------+-------------------------------------------
Best Regards,
Rajiv Duggal
Office: 805.388.1711 x 528
Mobile: 805.312.3499
Email: rduggal at dexsystems.com|<mailto:rduggal at dexsystems.com|> www.dex.com<http://www.dex.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/community/attachments/20160111/3083cc35/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: nova.conf
Type: application/octet-stream
Size: 2560 bytes
Desc: nova.conf
URL: <http://lists.openstack.org/pipermail/community/attachments/20160111/3083cc35/attachment-0004.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: haproxy.cfg
Type: application/octet-stream
Size: 4569 bytes
Desc: haproxy.cfg
URL: <http://lists.openstack.org/pipermail/community/attachments/20160111/3083cc35/attachment-0005.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: glance-api.conf
Type: application/octet-stream
Size: 55529 bytes
Desc: glance-api.conf
URL: <http://lists.openstack.org/pipermail/community/attachments/20160111/3083cc35/attachment-0006.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: glance-registry.conf
Type: application/octet-stream
Size: 44848 bytes
Desc: glance-registry.conf
URL: <http://lists.openstack.org/pipermail/community/attachments/20160111/3083cc35/attachment-0007.obj>
More information about the Community
mailing list